GlobalPointer: A Unified Way to Handle Nested and Non-nested NER

Basic Idea

Specifically, assume the input text sequence length is n. For simplicity, assume there is only one type of entity to identify, and each entity is a continuous segment of the sequence with no length limit, potentially nested (overlapping with other entities). How many "candidate entities" are there in this sequence? It is easy to see that the answer is n(n+1)/2. That is, a sequence of length n has n(n+1)/2 different continuous subsequences, which contain all possible entities. Our task is to pick the true entities from these n(n+1)/2 "candidate entities," which is essentially a "k-out-of-n(n+1)/2" multi-label classification problem. If there are m types of entities to identify, we perform m such multi-label classification problems. This is the basic idea of GlobalPointer: using the entity as the basic unit for discrimination, as shown in the image at the beginning of this article.

Some readers might ask: isn’t the complexity of this design clearly \mathcal{O}(n^2)? Won’t it be very slow? If this were the era of RNNs/CNNs, it might seem slow. However, in the current era where Transformers dominate NLP, every layer of a Transformer has \mathcal{O}(n^2) complexity. Adding or removing one GlobalPointer layer doesn’t change much. Crucially, \mathcal{O}(n^2) is only the space complexity; if parallel performance is good, the time complexity can even drop to \mathcal{O}(1), so there is no significant perceived delay.

Mathematical Form

Let the input t of length n be encoded into a vector sequence [\boldsymbol{h}_1, \boldsymbol{h}_2, \cdots, \boldsymbol{h}_n]. Through transformations \boldsymbol{q}_{i,\alpha} = \boldsymbol{W}_{q,\alpha}\boldsymbol{h}_i + \boldsymbol{b}_{q,\alpha} and \boldsymbol{k}_{i,\alpha} = \boldsymbol{W}_{k,\alpha}\boldsymbol{h}_i + \boldsymbol{b}_{k,\alpha}, we obtain vector sequences [\boldsymbol{q}_{1,\alpha}, \boldsymbol{q}_{2,\alpha}, \cdots, \boldsymbol{q}_{n,\alpha}] and [\boldsymbol{k}_{1,\alpha}, \boldsymbol{k}_{2,\alpha}, \cdots, \boldsymbol{k}_{n,\alpha}], which are used to identify entities of type \alpha. We can then define: s_{\alpha}(i,j) = \boldsymbol{q}_{i,\alpha}^{\top}\boldsymbol{k}_{j,\alpha} \label{eq:s} as the score for the continuous segment from i to j being an entity of type \alpha. That is, the inner product of \boldsymbol{q}_{i,\alpha} and \boldsymbol{k}_{j,\alpha} serves as the logit for the segment t_{[i:j]} being an entity of type \alpha. Under this design, GlobalPointer is essentially a simplified version of Multi-Head Attention, where each entity type corresponds to a head, but without the \boldsymbol{V}-related operations.

Relative Position

Theoretically, the design in Eq. [eq:s] is sufficient. However, in practice, when training data is limited, its performance is often suboptimal because it does not explicitly include relative position information. As we will see in later experiments, adding or not adding relative position information can result in a performance difference of over 30 percentage points!

For example, if we want to identify place names in a weather forecast like "Beijing: 21 degrees; Shanghai: 22 degrees; Hangzhou: 23 degrees; Guangzhou: 24 degrees; ...", there are many entities. Without relative position information, GlobalPointer is not particularly sensitive to the length and span of entities. Consequently, it might easily predict any combination of a start and end from different entities as a target (e.g., predicting "Beijing: 21 degrees; Shanghai" as an entity). Conversely, with relative position information, GlobalPointer becomes sensitive to length and span, allowing it to better distinguish true entities.

Which relative position encoding should be used? Theoretically, any relative position encoding used in Transformers could be considered (refer to "Transformer Positional Encodings that Make Researchers Rack Their Brains"). However, most relative position encodings involve truncation. While the truncation range is usually sufficient for NER, it feels somewhat inelegant. Not truncating would lead to too many learnable parameters. After consideration, the Rotary Positional Embedding (RoPE) I previously conceived seems most suitable.

RoPE is introduced in "Transformer Upgrade: 2. Rotary Positional Embedding". It is essentially a transformation matrix \boldsymbol{\mathcal{R}}_i satisfying \boldsymbol{\mathcal{R}}_i^{\top}\boldsymbol{\mathcal{R}}_j = \boldsymbol{\mathcal{R}}_{j-i}. Applying this to \boldsymbol{q} and \boldsymbol{k}, we get: s_{\alpha}(i,j) = (\boldsymbol{\mathcal{R}}_i\boldsymbol{q}_{i,\alpha})^{\top}(\boldsymbol{\mathcal{R}}_j\boldsymbol{k}_{j,\alpha}) = \boldsymbol{q}_{i,\alpha}^{\top} \boldsymbol{\mathcal{R}}_i^{\top}\boldsymbol{\mathcal{R}}_j\boldsymbol{k}_{j,\alpha} = \boldsymbol{q}_{i,\alpha}^{\top} \boldsymbol{\mathcal{R}}_{j-i}\boldsymbol{k}_{j,\alpha} This explicitly injects relative position information into the score s_{\alpha}(i,j).

Loss Function

So far, we have designed the score s_{\alpha}(i,j). Identifying entities of type \alpha becomes a multi-label classification problem with n(n+1)/2 classes. The key is the design of the loss function. The simplest approach is to treat it as n(n+1)/2 binary classifications. However, in practice, n is often not small, making n(n+1)/2 even larger, while the number of entities per sentence is small. This would lead to a severe class imbalance problem.

This is where our previous research "Generalizing ’Softmax + Cross Entropy’ to Multi-label Classification" comes in handy. Briefly, this is a loss function for multi-label classification that generalizes single-label multi-class cross-entropy, particularly suitable for cases with many total classes but few target classes. Its form is not complex; in the context of GlobalPointer, it is: \log \left(1 + \sum\limits_{(i,j)\in P_{\alpha}} e^{-s_{\alpha}(i,j)}\right) + \log \left(1 + \sum\limits_{(i,j)\in Q_{\alpha}} e^{s_{\alpha}(i,j)}\right) where P_{\alpha} is the set of (start, end) pairs for all entities of type \alpha in the sample, and Q_{\alpha} is the set of all non-entity pairs or entities not of type \alpha. Note that we only need to consider combinations where i \leq j: \begin{aligned} \Omega &= \big\{(i,j) \,\big|\, 1 \leq i \leq j \leq n\big\} \\ P_{\alpha} &= \big\{(i,j) \,\big|\, t_{[i:j]} \text{ is an entity of type } \alpha\big\} \\ Q_{\alpha} &= \Omega - P_{\alpha} \end{aligned} In the decoding stage, all segments t_{[i:j]} satisfying s_{\alpha}(i,j) > 0 are treated as entities of type \alpha. The decoding process is extremely simple, and with full parallelism, the decoding efficiency is \mathcal{O}(1)!

Evaluation Metrics

For NER, the common evaluation metric is F1, specifically entity-level F1, not label-level F1. Under traditional Pointer Network or CRF designs, it is not easy to calculate entity-level F1 directly during training. However, with GlobalPointer, calculating entity-level F1 or accuracy is straightforward. For example, F1 is calculated as follows:

def global_pointer_f1_score(y_true, y_pred):
    """F1 score designed for GlobalPointer
    """
    y_pred = K.cast(K.greater(y_pred, 0), K.floatx())
    return 2 * K.sum(y_true * y_pred) / K.sum(y_true + y_pred)

This simplicity is due to the "Global" nature of GlobalPointer; its y_true and y_pred are already at the entity level. By checking y_pred > 0, we know which entities were extracted, and matching them allows us to calculate various entity-level metrics, achieving consistency across training, evaluation, and prediction.

Optimizing F1 Score

Another advantage of GlobalPointer’s "Global" nature is that if we use it for Reading Comprehension (RC), it can directly optimize the RC F1 metric! RC F1 differs from NER F1 as it measures the fuzzy matching degree of the answer. Directly optimizing F1 may be more conducive to improving the final RC score. Using GlobalPointer for RC is equivalent to NER with only one entity type. We define: p(i,j) = \frac{e^{s(i,j)}}{\sum\limits_{i \leq j} e^{s(i,j)}} With p(i,j), using reinforcement learning ideas (refer to "Policy Gradient and Zero-order Optimization: Different Paths to the Same Destination"), optimizing F1 uses the following loss function: -\sum_{i\leq j} p(i,j) f_1(i,j) + \lambda \sum_{i\leq j}p(i,j)\log p(i,j) where f_1(i,j) is the pre-calculated F1 similarity between segment t_{[i:j]} and the standard answer, and \lambda is a hyperparameter. Calculating all f_1(i,j) might be costly, but it is a one-time operation and can be optimized (e.g., setting it to zero if the start and end are too far apart). Overall, it is manageable. This is a direct approach to try if the goal is to improve the final RC F1. (I tried this in this year’s Baidu LIC2021 RC track, and it did show some effect.)

People’s Daily

First, we verify whether GlobalPointer can replace CRF in flat scenarios using the classic People’s Daily corpus. The baseline is BERT+CRF, compared against BERT+GlobalPointer. The results are:

The most striking result is the gap between GlobalPointer with and without RoPE, which exceeds 30 percentage points! This demonstrates the importance of explicitly adding relative position information. In subsequent experiments, we will only use the version with RoPE.

The table also shows that in classic flat NER tasks, GlobalPointer’s performance is comparable to CRF, while its speed is superior. It is truly both fast and effective.

CLUENER

Since the People’s Daily task already has a high baseline, the gap might not be obvious. We also tested the newer CLUENER dataset, which is also flat. The current SOTA F1 is around 81%. The comparison is as follows:

These results show that as NER difficulty increases, even in flat scenarios, GlobalPointer outperforms CRF. This suggests that for NER, GlobalPointer is actually more effective than CRF. We will provide a theoretical analysis later to explain why GlobalPointer is more rational.

Regarding speed, since the text lengths in this task are generally short, the speed increase for GlobalPointer is not as significant.

CMeEE

Finally, we test a nested task (CMeEE), which was the "Chinese Medical Named Entity Recognition" competition on biendata last year and Task 1 of this year’s "CBLUE" challenge. It involves medical NER with some nested entities. We again compare CRF and GlobalPointer:

GlobalPointer significantly outperforms CRF. Regarding speed, across the three tasks, the longer the text, the more pronounced the training acceleration of GlobalPointer. Prediction speed is also slightly improved, though less so than training. I later experimented with RoBERTa large as the encoder and reached over 67% on the online test set, proving GlobalPointer is a "competent" design.

Some might argue that using a flat CRF for nested NER is an unfair comparison. This is true to an extent, but the impact is limited: CMeEE’s F1 is relatively low, and nested entities are not very frequent. Furthermore, I haven’t found a simple and clear nested NER design to use as a baseline, so CRF serves as a preliminary reference.

Comparison with CRF

CRF (Conditional Random Field) is a classic design for sequence labeling. Since most NER can be transformed into sequence labeling, CRF is a classic method. I have previously written "A Brief Introduction to CRF" and "Your CRF Layer’s Learning Rate Might Not Be Large Enough". As previously introduced, if the number of labels is k, the difference between frame-wise softmax and CRF is:

The former treats sequence labeling as n independent k-class classification problems, while the latter treats it as a single k^n-class classification problem.

This statement also highlights the theoretical shortcomings of both when used for NER. Frame-wise softmax is too loose; predicting a label correctly at one position doesn’t mean the entity is correctly extracted—the entire segment’s labels must be correct. Conversely, CRF is too strict; it essentially requires all entities in a sequence to be correct to get a "score." While CRF can produce partially correct results in practice due to the model’s generalization, its design implies an "all-or-nothing" approach.

Therefore, CRF has theoretical irrationalities. In contrast, GlobalPointer is closer to the actual usage and evaluation scenarios: it is entity-based and designed as a multi-label classification problem. Thus, its loss function and evaluation metrics are at the entity granularity, providing reasonable scores even if only some entities are correct. This is why GlobalPointer can outperform CRF even in flat NER scenarios.

Related Work

Readers following entity recognition and information extraction might notice that GlobalPointer is similar to the recent relation extraction design TPLinker. However, the idea of global normalization can be traced back further.

I first encountered this idea in Baidu’s 2017 paper "Globally Normalized Reader" (GNR), which proposed a global normalization design for RC. It treated (sentence, start, end) as a whole. Because the number of combinations was vast, it used ideas from "Sequence-to-Sequence Learning as Beam-Search Optimization" to reduce computation.

With GNR as a precursor, GlobalPointer is a natural progression. I had similar ideas during the LIC2019 relation extraction track, but several issues were unresolved then.

First, Transformers were not yet popular, and \mathcal{O}(n^2) complexity seemed daunting. Second, the multi-label cross-entropy loss hadn’t been developed, so class imbalance was a major hurdle. Third, my understanding of NLP was shallower, and bert4keras didn’t exist, making experimentation difficult. (For instance, if I had missed RoPE two years ago and saw a 30-point drop, I wouldn’t have known how to fix it.)

Thus, GlobalPointer is a "coincidental" yet "inevitable" result of two years of accumulation. While similar in form to TPLinker, GlobalPointer’s origin is independent. TPLinker itself can be traced back to "Joint entity recognition and relation extraction as a multi-head selection problem", though that series focused on relation extraction rather than optimizing for NER.

Additive vs. Multiplicative

In implementation, a major difference between TPLinker and GlobalPointer is that TPLinker uses additive attention for Multi-Head: s_{\alpha}(i,j) = \boldsymbol{W}_{o,\alpha}\tanh\left(\boldsymbol{W}_{h,\alpha}[\boldsymbol{h}_{i},\boldsymbol{h}_{j}]+\boldsymbol{b}_{h,\alpha}\right)+\boldsymbol{b}_{o,\alpha} It is unclear how much the performance differs from Eq. [eq:s]. However, compared to the multiplicative attention in Eq. [eq:s], additive attention is much more computationally expensive, especially in terms of memory (VRAM), despite similar theoretical complexity.

I believe that even if additive attention performs slightly better, optimization should continue on the multiplicative basis because additive efficiency is poor. Furthermore, TPLinker and similar papers did not report the importance of relative position information; it is unknown if relative position is less critical in additive attention.