Fast Labeled Spanning Tree in Binary Irregular Graph Pyramids

: Irregular Pyramids are powerful hierarchical structures in pattern recognition and image processing. They have high potential of parallel processing that makes them useful in processing of a huge amount of digital data generated every day. This paper presents a fast method for constructing an irregular pyramid over a binary image where the size of the images is more than 2000 in each of 2/3 dimensions. Selecting the contraction kernels (CKs) as the main task in constructing the pyramid is investigated. It is shown that the proposed fast labeled spanning tree (FLST) computes the equivalent contraction kernels (ECKs) in only two steps. To this purpose, first, edges of the corresponding neighborhood graph of the binary input image are classified. Second, by using a total order an efficient function is defined to select the CKs. By defining the redundant edges, further edge classification is performed to partition all the edges in each level of the pyramid. Finally, two important applications are presented : connected component labeling (CCL) and distance transform (DT) with lower parallel complexity 𝒪( 𝑙𝑜𝑔 ( 𝛿 )) where the 𝛿 is the diameter of the largest connected component in the image.


Introduction
Pyramids are important structures in pattern recognition and image processing. They were invented [1] as ordered collection of images at multiple resolutions that are able to process high resolution data at lower resolution and propagating the local information into global and abstracted information at higher levels [2]. Pizlo [3] states that the pyramid is a general model for human problem solving where a massively parallel processing must be accomplished in order to recognizing a complex scene (like a busy street) in the blink of an eye [4,5].
Motivated by a biological point of view, this paper introduces a fast method to construct the pyramidal structure of a given 2D binary image in a fully parallel scheme. Using the built pyramid, fundamental operations in analysing the binary images can be performed with lower complexity: Connected Component Labeling (CCL) and Distance Transform (DT). In particular, the current research is an extension of the previous work [6] that computes connected components (CCs) with the help of the pyramid. Propagating the labels in [6] is performed in linear time, hence the parallel complexity at the worst case is ( ) where is the diameter of the largest CC in the image. In contrast, this paper mathematically proves that the parallel complexity is decreased to ( ( )). The paper is organized as follows. Sec. 1 gives a short overview of the theoretical background of image pyramids, graph pyramids and different graph representations. The classification of edges is defined in Sec. 2. Selecting the contraction kernels as the main step in constructing the irregular pyramid is completely described in Sec. 3. To this aim, the concept of redundant edges is covered by detail. The proposed fast labeled spanning tree (FLST) is defined in Sec. 4. Two main applications are presented in Sec. 5. The last section, provides a conclusion and considerations for future research.

Image Pyramids
Image Pyramids consist of a series of successively reduced images produced from a high resolution base image [2]. Generally, two types of the pyramids, namely regular and irregular pyramids exist. In regular pyramids [7] the resolution is decreased in regular steps and therefore the size of the pyramid is fixed. On the contrary, in irregular pyramids [8,9] the size of the pyramid is not fixed and it is adapted to the image data. In addition, unlike the regular ones, the irregular pyramids are shift-and rotation-invariant which make them useful to use in a variety of tasks, in particular image segmentation [10,11].
It should be noticed that the irregular image pyramid is interpreted as the irregular graph pyramid when its pixels and the neighborhood relations between adjacent pixels correspond to the vertices and the edges of the graph, respectively. below by selecting a specific subset of vertices and edges. For generation of irregular pyramids, two basic operations on graphs are needed: edge contraction and edge removal. The former contracts an edge connecting two vertices, and the two vertices are joined into one. All edges that were incident to the joined vertices will be incident to the resulting vertex after the operation. The latter removes an edge from the graph, without changing the number of vertices or affecting the incidence relationships of other edges.
In each level of the pyramid, the vertices/edges which disappear in a level above are called non-surviving vertices/edges. Those vertices/edges which appear in the upper level are called surviving vertices/edges. Consider = ( , ) as the neighborhood graph of an image where corresponds to the vertex set and corresponds to the edge set. The vertex ∈ associates with the pixels in image and the edge ∈ connects the corresponding adjacent vertices. Let the gray-value of vertex ( ) = ( ) where ∈ is a pixel in the image corresponding to vertex . Consider ( ) as an attribute of an edge ( , ) where , ∈ and ( ) = | ( ) − ( )| in the base level. Since we are working with binary images only, the vertices have either of the two values 0 and 1. Similarly the contrast of an edge is either 0 or 1.

Definition 1 (Contraction Kernel (CK)).
A CK is a tree consisting of a surviving vertex as its root and some non-surviving neighbors with the constraint that every non-survivor can be part of only one CK.
An edge of a CK is denoted by the directed edge and points towards the survivor. In this paper, the 4-connectivity between pixels of the input image is assumed. The reason is that the 8-connectivity would not be a plane graph [12]. A plane graph is a graph embedded in the plane such that its edges intersect only at their endpoints [13]. In a plane graph there are connected spaces between edges and vertices and every such connected area of the plane is called a face. The degree of the face is the number of edges bounding the face. In addition a face bounded by a cycle is called an empty face. In a non-empty face, traversing the boundary would require to visit vertices or edges twice [12]. An empty face consisting only one edge is called an empty self-loop. Consider an empty face of degree 2: it contains two edges that have the same endpoints. These parallel edges are called multiple edges. The multiple edges mean edges between the same endpoints, i.e. for example edges 1 , 1 ≠ 2 , 2 ≠ 3 , 3 where 1 = 2 = 3 and 1 = 2 = 3 .

Graph Representation
Graphs as a versatile representative tool are common in the representation of the irregular pyramid. There are different graph representations such as a simple graph, a dual graph and a combinatorial map.
A simple graph [14] = ( , ) consists of a set of vertices and of edges without self-loops and multiple edges between pairs of vertices. The relationships between different regions can be represented by the region adjacency graph (RAG). Although plane simple graphs are a common model for the RAG they cannot distinguish between different topological configurations, namely inclusion and multiple adjacency relationships (multi-boundaries) of regions [14]. A dual graph model encodes multiple boundaries between regions in a non-simple graph. The problem with dual graphs [9] is that they cannot unambiguously represent a region enclosed in another one on a local level [14]. Therefore, in this paper the combinatorial map (CM), as a planar embedding of a RAG, is used. It not only solves the mentioned problems but also provides an efficient structure to preserve topological relations between regions while it can be extended to higher dimensions (nD).

Combinatorial Pyramid
A combinatorial pyramid [15] is a hierarchy of successively reduced combinatorial maps. A combinatorial map (CM) is similar to a graph but explicitly stores the orientation of edges around each vertex. The 2D combinatorial map ( ) is defined by a triple = ( , , ) where the D is a finite set of darts [14]. A dart is defined as a half edge and it is the fundamental element in the CM's structure. The is an involution on the set D and it provides a one-to-one mapping between consecutive darts forming the same edge such that ( ( )) = . The is a permutation on the set D and encodes consecutive darts around the same vertex while turning counterclockwise [16]. Note that the clockwise orientation is denoted by −1 . Fig. 1a shows a set adjacent darts with their and encoding. Note that the edge between two vertices and is denoted by = ( , ( )). The , ∈ and the ∈ where the and are the set of vertices and edges of the graph = ( , ), respectively. The removal and the contraction operations in the combinatorial pyramid is defined as follows: Definition 2 (Removal operation). The removal operation removes one edge, \{ }, while it modifies the adjacent darts such that: Definition 3 (Contraction operation). The contraction operation removes one edge, /{ }, and collapses its two endpoints and modifies the adjacent darts such that: Fig. 1b and Fig. 1c illustrate the removal and contraction operations in the combinatorial map. Note that the contraction operation does not disconnect the graph, and thus preserves connectivity [8].

Edge Classification in a Binary Image Graph
Let neighborhood graph = ( , ) be the undirected connected plane graph consisting of a finite set of vertices and a finite set of edges . In the neighborhood graph of the binary input image, each connected component (CC) consists of a set of vertices with the same gray value, 0 or 1. In the paper, black pixels (vertices) are shown by 0 while white pixels (vertices) are shown by 1   partition the edges of the neighborhood graph into two categories: edges connecting two vertices of the same CC, intra-CC and edges connecting vertices of different CCs, inter-CCs as follows:

Definition 5. Inter-CC edge: an edge
The contrast of an intra-CC edge is equal to zero, ( -) = 0. Therefore, we denote the intra-CC edge by 0 ∈ 0 . The contrast of an inter-CCs edge is one, ( -) = 1. Therefore, the inter-CCs edge is denoted by 1 ∈ 1 . All edges in the neighborhood graph are partitioned into 0 and 1 edges:

Selecting the CKs using a Total Order
Selecting the CKs plays the main role in constructing the irregular pyramid. The height of the built pyramid and the complexity of the construction depends on how the CKs are selected. In order to achieve an efficient and a unique selection of the CKs a total order is defined over the vertices [17]. Consider as the neighborhood graph of an binary input image with by vertices. Let (1, 1) be the coordinate of the vertex at the upper-left corner and ( , ) at the lower-right corner. Let and denote the row and the column in the grid structure of , respectively. The vertices of receive a unique index as follows: We use the properties of the total order [18] in selecting the CKs. First, every two elements of a total ordered set (indices of vertices) are comparable. Second, each subset of the total ordered set (a set of vertices) has exactly one minimum and one maximum. In the binary neighborhood graph a CC consists of only intra-CC ( 0 ) edges. In constructing the irregular pyramid this CC is shown by only one single vertex at the top of the pyramid. Therefore, all the CKs are selected only from the intra-CC edges. From the vertex point of view, a vertex that is not incident to an intra-CC edge is an isolated vertex. This vertex is surrounded by only inter-CC edges.
Let be a non-isolated vertex, i.e, it is the endpoint of at least one intra-CC edge. The upper neighborhood is defined as follows : The cardinality of the set | ( )| indicates the number of intra-CC edges incident to having greater vertex than . Therefore, the cardinality of the non-isloated vertex is | ( ) ≥ 1|. In order to determine the CKs in the graph = ( , ), the selecting contraction kernel SCK(.) function is defined as follows: The output of the SCK function partitions the vertices into two categories: surviving vertices and non-surviving vertices as follows:

Proposition 1. An isolated vertex survives always.
Proof. Assume is an isolated vertex. Since there is no intra-CC edge incident to the isolated vertex, it leads to | ( )| = 0. Based on the (9), ( ( )) = ( ) and therefore employing the Def. 6 is the surviving vertex.  Proof. Based on (5) a non-surviving vertex may be incident to maximum two vertices with greater indices (right or down vertices) at the base level. In addition, the non-surviving vertex must be incident to at least one vertex, namely its right or its down vertex. The former states | ( )| = 2 while the latter states | ( )| = 1.

□
In a CK there is one surviving vertex (the root of the CK) while the remaining vertices are non-surviving vertices. Each non-surviving vertex connects to the surviving root by a unique monotonically -increasing path of oriented edges. In a graph with vertices there are ! different total order1. Each selected total order has its own properties. Selecting an efficient total order effects on selecting the CKs where the number of CKs determines the height of the pyramid. Pyramids with logarithmic height reduce the parallel computational complexity of fundamental operations such as connected component labeling [19,17] and distance transform [20]. Therefore, a proper selection of the total order must result in constructing the pyramid with logarithmic height. In Sec. 4.1 it is proved that the proposed total order leads to this logarithmic height. In contrast to the common methods of constructing the pyramid [2,8], using the proposed total order has the advantage that the vertices are partitioned in every level of the pyramid. In other words, the vertices are either the non-surviving or the surviving vertices. Next sections show how this partitioning reduces the number of steps in selecting the CKs into only two steps.

First Step of Selecting the CKs
Selecting the CKs at the base level of the pyramid is the first step of the selection. To this aim, the SCK function is performed over each vertex of the neighborhood graph of the base level. As the result, each CK has one surviving vertex and all the other vertices of the CK do not survive. In Fig. 2

Redundant Edges
Graphs as a versatile representative tool may have many unnecessary (redundant) edges [17]. Through the construction of the pyramid, contracting edges results in a smaller induced graph at the upper level. The resulting graph may consist of empty self-loops or double-edges. At this point, the edge removal simplifies the graph and removes these redundant edges.

Definition 8 (Redundant Edges). In a hierarchical structure, those edges that are not needed to fully reconstruct the hierarchy are considered as redundant edges.
Generally, the definition of the redundant edges depends on the applications and to what extend the reconstruction needs to be performed. For example, Banaeyan et al. [6,17,19] defined the concept of the redundant edges in a binary graph pyramid in order to do the connected component labeling task where the fully reconstruction is performed. They showed that the redundant edges can be detected (predicted) before performing the contraction of edges. In this paper, we use the same concept for defining the redundant intra-CC and redundant inter-CC edges. Definition 9 (Redundant Intra-CC edge ( 0 )). In an empty face consisting of only intra-CC edges, the non-oriented edge incident to the vertex with lowest Idx is a redundant intra-CC edge.
The definition above states that a redundant intra-CC edge ( 0 ) exists only in a face bounded by intra-CC edges. Fig. 3 illustrates the configuration of the redundant intra-CC edges.

Definition 10 (Redundant Inter-CCs Edge ( 1 )). In an empty face, an inter-CCs edge incident to the vertex with lowest Idx is redundant iff:
• The empty face consists of only two inter-CCs edges.
• The empty face is bounded by inter-CCs edges and oriented intra-CC edges.

Second Step of Selecting the CKs
At the base level of the pyramid there are three types of the intra-CC edges: 1. The oriented edges that belong to the CKs.
3. The remaining non-oriented intra-CC edges are defined as the bridges.

Definition 11 (Bridge). A bridge is a non-oriented intra-CC edge that bridges the gap between two contraction kernels of a connected component.
Note that the bridge is the edge of the equivalent contraction kernel (ECK) that is contracted after the two CKs are contracted.

Proposition 4. A bridge in a face of degree 4 at the base level of the pyramid is the vertical edge.
Proof. In the face of degree 4, there are two horizontal and two vertical edges. Assume the non-oriented bridge is the horizontal edge. However, due to Pro. 3 every horizontal intra-CC edge is oriented and therefore it cannot be a non-oriented intra-CC edge. Proof. Assume a face of degree 4 contains two bridges. Since the bridges are vertical intra-CC edges, the oriented intra-CC edge must connect two different CCs which is in contradiction with the definition of the oriented edge (see Fig. 5  Proof. Assume that two bridges are incident to the same vertex. Therefore, the horizontal common edge between their two corresponding faces must be the oriented intra-CC edge and the inter-CCs edge at the same time (see Fig. 5-c) , contradiction. In order to select the CKs at the second step, the SCK function is performed over the bridges.  Note that if the inclusion relationships exists between two different CCs, the inclusion edge is one of the bridges that will be detected after the contractions of oriented edges. In Fig. 2 the inclusion edge is shown by a non-empty selfloop that is denoted by the letter a with the red color.

Proposition 7. All the redundant intra-CC edges are detected at the base level of the binary pyramid.
Proof. The redundant intra-CC edges occur in a face bounded by only intra-CC edges. At an upper level, the remaining intra-CC edges are the bridges at the base level. However, since each bridge has the -related to a inter-CCs edge (Fig. 6a), therefore, there is no empty face containing only intra-CC edges at upper levels of the pyramid. Note that in the simple graph , two bridges cannot be therelated of each other because this contradicts to planarity of the graph (Fig. 6b and Fig. 6c) or it contradicts to (5) that is shown in Fig. 6d.

□
The Proposition .7 states that there is no redundant intra-CC edge at an upper level of the pyramid. In fact, this is because of the important property of the defined total order over the indices of vertices where at the base level each non-surviving vertex only can be contracted into its right or down neighborhood vertex.

Fast Labeled Spanning Tree (FLST)
A CC in a binary graph pyramid is represented by a single surviving vertex at the top level of the pyramid. This vertex is the root of the tree spanning its receptive field at the base level [21]. In [22] it was shown that the combination of two (or more) successive reductions in an equivalent weighting function allows to calculate any level of the pyramid directly from the base. Kropatsch in [21] introduced the Equivalent Contraction Kernels (ECK) in the irregular graph pyramid and it was later used [23] in the minimum spanning tree (MST) segmentation.
In the binary pyramid, every spanning tree of a CC is the MST because the contrast (weight) of the intra-CC edges is zero. To drive the spanning tree of a CC, the previous common methods [14,16,11] need to select the CKs in iterations where is the height of the pyramid. In contrast, in the proposed method we only need two steps of selecting the CKs. Moreover, the SCK function is performed locally over each vertex. This means that the CKs are selected with parallel complexity of (1). Note that, it is assumed there are sufficient processing elements available in order to do the parallel computations.

Independent Edges
To contract the CKs in a parallel manner, finding a set of independent edges plays the key role. Dependency of the edges differs based on what processing is going to be performed between a set of edges. In [19] two edges not sharing an endpoint are considered as independent edges. Using this definition all the CKs at the first selection can be contracted with parallel complexity bounded as follows: ( 2 ( ( ))) ≤ ≤ ( 3 ( ( ))) (10) To determine the parallel complexity of contracting the CKs at the second step of selection, the dependencies between darts [6] is considered. Since in this step, each edge of the CK is a bridge at the base level, hence, there is an inter-CCs edge with a -relation incident to this edge. Therefore, all the CKs at the second selections are independent of each other and they will be contracted in parallel complexity (1).

Applications
To highlight the usefulness of the proposed method, two main applications are presented. In both application the parallel complexity is ( ( )) in a × -size input binary image.

Connected Component Labeling
Connected Component Labeling (CCL) is a fundamental task in analyzing binary images [24] where background and foreground are denoted by zero and one, respectively. A connected region is a group of pixels where all pairs of pixels are connected together. The role of the CCL is to assign a unique label to each CC. Common methods of CCL [24,25,26] are linear; i.e., they search the binary image row by row in the raster-scan fashion. In contrast, using a hierarchical structure, within the bottom-up construction each pixel reaches its single surviving pixel (super-pixel) at top of the pyramid in a logarithmic number of steps. At this top-level of the pyramid, each of the super-pixels receives its unique label . Afterwards, through the top-down propagation the vertices of the lower levels inherit the labels from the higher levels until all the pixels at the receptive field (base level of the pyramid) received their labels. The hierarchical method is called Parallel Pyramidal Connected Component (//ACC2) where the details can be found in [19]. The //ACC not only does the CCL task but also preserves the topological relations between the CCs. Fig. 7 shows how the //ACC encodes inclusion relationships between three CCs. Table 1 [19] shows the execution time of the //ACC method over three different categories of binary images; Random, MRI and Finger-print images. In addition, the execution time of the algorithm over different image-size is compared to the state-of-the-art methods; Spaghetti_RemSP, BBDT_RemSP, SAUF_UF, in [27]. The results in Fig. 8 encourage the //ACC method should be used in large images including more than one million pixels.

Distance Transform
The distance transform (DT) is another important fundamental operation that is applied to the binary image [1]. It is employed in a broad range of applications containing template matching [29,30], image registration [31], map matching robot self-Localization [32], skeletonization [33], Line Detection in Manuscripts [34], Weather Analysis and Forecasting [35], etc. After applying the DT to a binary image, the result of the transform is a new gray-scale image whose foreground 1 pixels have intensities representing the minimum distance from the background 0 pixels. In order to compute the DT, the common methods [1,36], propagate the distances in linear sequential time. By contrast, using the hierarchical structure the distances can be propagated by a set of power-of-two numbers [20] where the parallel complexity is reduced into the logarithmic-time. The computation of DT with lower complexity makes the pyramid as a useful tool in analysing large binary images. In particular, currently we are working on the Water's gateway to heaven project3 dealing with high-resolution X-ray microtomography ( ) and fluorescence microscopy. The size of the images is more than 2000 in each of 3 dimensions where we use the saddle points of the DT to separate cells, which are visually difficult to be separated.
In the mentioned project above the input image is a labeled 2D cross slice of a leaf scan where it has six different labels illustrating different regions inside the leaf (Fig. 9a). The task of stomata is to control the amount of CO 2 that is entering the leaf. In order to do the photosynthesis, the CO 2 propagates through the airspace inside the leaf to reach the cells where it combines with water and sunlight. To model the procedure of the gas exchange in the leaf [37], we compute the geodesic distance transform (GDT) from the stomata through the airspace (Fig. 9b) to find out how long it takes to reach the necessary CO 2 concentration [20]. The use of pyramids would enormously speed up the computations of the DT in the large images of the project.

Conclusion
The paper presents a fast parallel method to select the equivalent contraction kernels in the irregular pyramid of a binary input image. It was shown that the first step of selecting the contraction kernels (CKs) at the base level is done with parallel complexity (1). These CKs are contracted with parallel ( ( )) complexity where the is the diameter of the maximum connected component (CC) in the neighborhood graph of the image. By detecting the redundant edges (RE) the selection of CKs is performed in one parallel step. By defining the independent set of edges, we proved that all the selected CKs at the second step of selection are contracted in parallel complexity (1). The Fast labeled spanning tree (FLST) of the CCs is produced with parallel complexity ( ( )). Using the total order there is no random processing in construction of the pyramid and the resulting FLST is unique.
In addition, it was shown by employing the proposed FLST, that the fundamental operations in analyzing the binary image can be performed in lower parallel complexity. In particular, two main operations, connected component labeling (CCL) and distance transform (DT), were presented in detail. Finally, we presented how the proposed method can be useful in processing of the large images in practical real applications. For future works we plan to compute 3D distance transform in order to study the diffusion in the air space within a leaf. WALTER G. KROPATSCH. From 1990-2021 he was full professor at TU Wien. He received his diploma degree in Technical Mathematics from the Technical University in Graz. He then moved to Grenoble, France to get the Maître d'Informatique from the University of Grenoble. His PhD in 1982 was on the Registration of Satellite Images with Maps. In 1984 he was invited by Prof. Azriel Rosenfeld to spend a year at the Center for Automation Research of the University of Maryland.
The creation of the first group in Austria dealing with pattern recognition and image processing in 1990 was jointly coordinated with the Austrian Association of Pattern Recognition (AAPR), that he initiated and led from 1984 until 1995. Under his leadership the AAPR became a member of the International Association of Pattern Recognition (IAPR) in which he held several leading positions, 2004-2006 he was its president. In 1996 he organized the main conference of the IAPR, the International Conference of Pattern Recognition in Wien, Austria. His scientific research focuses on pyramidal representations of images since his collaboration with Azriel Rosenfeld in 1984/85. The current graph-based pyramids follow similar concepts with the advantage that graphs are much more flexible data structures than the regular grids as currently used as architectures in deep learning. In his more than 400 scientific contributions many other concepts and applications have been addressed. He is currently senior editor of the journal of Electronic Imaging, and associate editor the journal of the Visual Computer and of several special issues in Pattern Recognition and Pattern Recognition Letter.