After the first part, we have successfully extracted the text features of the image. Next, we proceed with text localization. The main process is divided into two steps: 1. Proximity search, with the goal of circling single lines of text; 2. Text segmentation, with the goal of cutting single lines of text into individual characters.
Proximity Search
We can perform a connected component search on the extracted feature map, treating each connected component as a Chinese character. This is applicable to most Chinese characters, but it does not work for some simpler characters, such as "Xiao" (small), "Dan" (dawn), "Ba" (eight), and "Yuan" (currency). Because these characters lack connectivity, they are split into multiple parts, as shown in Figure 13. Therefore, we need a proximity search algorithm to integrate regions that likely form a single character or line, obtaining single-line text regions.
The purpose of proximity search is to perform dilation to "glue" together regions that may belong to the same character or line. If dilation is performed without searching, it occurs in all directions simultaneously, which might glue upper and lower lines together. Therefore, we only allow the region to dilate in a single direction. We use proximity search to determine the dilation direction (up, down, left, or right):
Proximity Search (Preliminary): Starting from a connected component, find its horizontal bounding box and expand the connected component to the entire rectangle. When the distance between this region and its nearest neighbor is within a certain range, consider dilating this rectangle. The direction of dilation is the direction where the nearest neighbor is located.
Since proximity is involved, we need a definition of distance. Below is a reasonable definition of distance.
Distance
As shown in the figure above, a rectangular region can be determined by the top-left coordinates (x, y) and the bottom-right coordinates (z, w), where the coordinates are calculated with the top-left corner as the origin. The center of this region is \left(\frac{x+w}{2}, \frac{y+z}{2}\right). For the two regions S and S' in the figure, we can calculate the difference between their center vectors: (x_c, y_c) = \left( \frac{x' + w'}{2} - \frac{x + w}{2}, \frac{y' + z'}{2} - \frac{y + z}{2} \right) \tag{10} It is unreasonable to use \sqrt{x_c^2 + y_c^2} directly as the distance because proximity should be calculated based on boundaries rather than center points. Therefore, we need to subtract the lengths of the regions: (x'_c, y'_c) = \left( x_c - \frac{w - x}{2} - \frac{w' - x'}{2}, y_c - \frac{z - y}{2} - \frac{z' - y'}{2} \right) \tag{11} The distance is defined as: d(S, S') = \sqrt{[\max(x'_c, 0)]^2 + [\max(y'_c, 0)]^2} \tag{12} As for the direction, it can be determined by the argument (angle) of (x_c, y_c).
However, according to the "Proximity Search (Preliminary)" method mentioned earlier, it is easy to glue the upper and lower lines of text together. Therefore, based on our horizontal layout assumption, a better method is to only allow horizontal dilation:
Proximity Search: Starting from a connected component, find its horizontal bounding box and expand the connected component to the entire rectangle. When the distance between this region and its nearest neighbor is within a certain range, consider dilating this rectangle. The direction of dilation is the direction where the nearest neighbor is located, and the dilation operation is executed if and only if the direction is horizontal.
Results
With the distance defined, we can calculate the distance between every two connected components and find the nearest neighbor for each. We expand each region by one-fourth in the direction of its nearest neighbor. In this way, adjacent regions may merge into a new region, thereby integrating fragments.
Experiments show that the proximity search approach can effectively integrate text fragments. The results are shown in Figure 15.
When reprinting, please include the original address of this article: https://kexue.fm/archives/3818
For more detailed reprinting matters, please refer to: "Scientific Space FAQ"