Medical image registration utilizing tissue P systems

The tissue P system (TPS) possesses intrinsic attributes of parallel execution in comprehensive data and instruction space, which provides fast convergence during the transition from local to global optima. Method- In this study, we have proposed and built a TPSysIR framework using the TPS for image registration that optimizes upon the mutual information (MI) similarity metric to find a global solution. Result- The model was tested on single- and multimodal brain MRI scans and other prominent optimization-based image registration techniques. Conclusion- Results show that, among all methods, TPSysIR provides better MI values with minimum deviation in a range of experiment setups conducted iteratively.

Introduction

Medical image registration involves processing image data from multiple sources, each having a different coordinate system. These sources often have different sensors and viewpoints, transforming the data collected into a single spatial coordinate system. Image registration requires optimized parameter values for the required transformation, translation, or rotation to be applied over the source images with respect to the reference image to achieve matching. Image registration has been utilized in many recent advances in image reconstruction (Prakash et al., 2019), land cover mapping (Wang et al., 2020), and weather prediction (Kakimoto et al., 2019). A large number of image registration methods are already available, which can be classified as single or multimodal, automatic vs. inter-active, spatial domain vs. frequency domain-based, intensity vs. feature-based, and transform-based. Image registration finds essential applicability in the areas of remote sensing and medical image processing. Image registration can be viewed as an optimization problem (Song et al., 2017) whose aim is to maximize the similarity or minimize the cost in the process. It takes single or multiple image data and transforms them according to the parameters to maximize the similarity to the target reference image. Various parameters which can be optimized are correlation ratio (Gong et al., 2019), mutual information (Ramamoorthy et al., 2010), energy of joint probability distribution (Susskind et al., 2011), normalized correlation (Lewis, 2001), and normalized mutual information (Knops et al., 2005).

Optimization methods such as Powell’s were among the earliest attempts to solve image registration problems. However, the algorithm provided local optimum results, and the objective search speed was also low. These shortcomings lead to the utilization of nature-based optimization techniques for image registration as the next-generation solution. The genetic algorithm (GA)-based method has been proposed by Rouet, Jacq, and Roux (Rouet et al., 2000). Utilizing the local optima, LI Zuo-zhu (Zuo-zhu, 2007) applied GA on mutual information (MI) metric optimization to achieve an image registration solution. Rajapakse and Guojun (Rajapakse and Guojun, 1999) performed image registration by utilizing GA on time-series images. Still, the nonexistence of fine-tuning ability coupled with considerable execution time led researchers to explore better methods. Chen, Lin, and Mimori (Lin et al., 2012) utilized particle swarm optimization (PSO) on the image dataset, optimizing the MI measure. Wachowiak et al. (Wachowiak et al., 2004), Chel, and Nandi (Chel and Nandi, 2013) used hybrid PSO on 3-D medical images; similarly optimization was performed on normalized MI decreasing the overall execution time. Basset et al. (Abdel-Basset et al., 2017) utilized modified MI metric and PSO for image registration. This method fell into local maxima with increasing degrees of rotation. Zhang et al. (Zhang et al., 2010) integrated PSO with Powell to overcome these shortcomings and applied them to image registration.

In recent years, machine learning (Zhu et al., 2022) and deep learning (Zhu et al., 2021) have found applications in image processing and registration. Balakrishna et al. (Balakrishnan et al., 2019) used a convolutional neural network for 3D image registration. Ali and Rittscher (Ali and Rittscher, 2019) utilized concatenated convolutional layers for deformable image registration. Mansilla, Milone, and Ferrante (Mansilla et al., 2020) proposed the AC-RegNet architecture to achieve image registration.

Membrane computing (MC) (Paun, 2000) was introduced by Gheorghe Paun, inspired by the computational mechanisms of living cells or tissue systems. Biological and computational processes at the cellular and tissue level are performed in a maximally parallel and randomly distributed manner. These random processes and communications are triggered when appropriate compounds and catalyst inhibitors are present in the cellular environment. Membrane computing forms the computational model called P systems; these have been efficiently utilized to obtain solutions to many NP-complete (Paun, 2001) problems by creating a trade-off between time and space complexity. P systems are built upon low-level biological interactions or processes by equipping them to capture the computational essence of complex cell metabolism and information interchange. The P system may use any one of the following mechanism or mechanisms to create variants of the system: selective object recognition, controlled exchange of particles through protein channels, cytoplasmic metabolism or division, and dissolution of membranes. The P system has been proved computationally complete and is utilized to solve many optimization-based and NP-complete problems (Paun, 2001), such as subset sum (Jiménez and Núñez, 2005), TSP, and tricolor problems. Computation in a P system proceeds in a maximally parallel and non-deterministic path, which can be tuned according to the execution model of the problem.

Membrane system

The membrane system shown in Figure 1 can be viewed as a hierarchically organized set of membranes existing inside an outer space called the environment. The tree in Figure 2 can represent the hierarchical organization of a membrane system in Figure 1. The tree’s root is associated with the skin membrane, and the leaves are associated with the elementary membranes. The membranes at the same level can float around in the same membrane compartment. The hierarchical string expression 1) is written for Figure 2 membrane structure:

${\left[\right.}_1{\left[\right.}_2{\left.\right]}_2{\left[\right.}_3{\left.\right]}_3{\left[\right.}_4{\left[\right.}_5{\left.\right]}_5{\left[\right.}_6{\left.\right]}_6{\left.\right]}_4{\left.\right]}_1.(1)$

FIGURE 1

FIGURE 1. Membrane system.

FIGURE 2

FIGURE 2. Membrane hierarchy.

P systems

The operations of a P System can be visualized as an extended distributed computation machine that presents a range of solutions to a particular problem. The nature of the multiset solution present in the output environment or membrane varies, based on the halting condition associated with the problem. The distribution and transition of multisets in the membrane regions determine the generated languages and their related grammar.Formally a P system ∏ can be defined as

$\prod =\left(O,M,m_1,m_2,\dots m_n,R_1,R_2,\dots ,R_n,{\delta }_0\right).(2)$

Here, O is the finite set of objects. M is the set of membranes. m_i is the multiset of objects in the membrane. R_i is the rule inside the corresponding membrane. δ₀ is the set of output membrane.

A P System can be viewed as a hierarchical system comprising of a three-dimensional space referred environment containing membranes. The membranes contain a set of objects called multiset, coexisting with rules and other membranes.Figure 3 shows a P System.

FIGURE 3

FIGURE 3. P system.

The P System can be formally written as

$\begin{array}{c}\hfill \prod =\left(\left\{a,b,c,d,,e\right\},\left\{\mathrm{1,2,3,4,5,6}\right\},\left\{,ac,c{d}^2,a{b}^{2}c,b^2{c}^2\right\}\right.\hfill \\ \hfill \left\{\left\{e\to e_{\mathit{out}},\right\}\left\{ac\to ab,a\to b_{\mathit{out}}\right\},\left\{cd\to e_{\mathit{out}}\right\},\right.\hfill \\ \hfill \left\{bc\to e_{\mathit{out}}\right\},\left\{bc\to cd,ac\to \#\right\},\left\{bc\to e_{\mathit{out}}\right\},\hfill \\ \hfill \left.\left.\left\{bc\to a,a\to \#\right\}\right\},\left\{1\right\}\right)\hfill \end{array}.(3)$

Here, O = {a, b, c, d, e}, M = {1, 2, 3, 4, 5, 6}, and m_i = $\{,ac,c{d}^2,a{b}^{2}c,b^2{c}^2\}\begin{array}{ccc}{R}_i\hfill & \hfill =\hfill & \left\{\{e\to e_{\mathit{out}},\},\{ac\to ab,a\to b_{\mathit{out}}\},\right.\hfill \\ \hfill & \hfill \hfill & \{cd\to e_{\mathit{out}}\},\{bc\to e_{\mathit{out}}\},\left\{\right.bc\to cd,\hfill \\ \hfill & \hfill \hfill & ac\to \#\left.\right\},\left.\{bc\to a,a\to \#\}\right\}\hfill \end{array}{\delta }_{0=\left\{1\right\}}$.The rules in the membranes are represented as x → y or x → y#, with x ϵ O⁺ and v ϵ (O × Tar)*, where Tar = {here; in; out}. Many forms of multiset rewriting and communicating rules have been utilized to convey more information about the state. There are mainly two types of rules: evolution and communication rules. The evolution rules govern the evolution of the membrane state, and communication rules facilitate communication, i.e., data transportation from one membrane to another. Evolution rules are of the form l → m or l → #; here, the occurrence of the # symbol leads to the dissolution of the membrane wherever the rule is executed, and all multisets currently existing inside it are passed onto its parent membrane. Communication rules are of form w → x_outy_in; here the multiset w forms two multisets, x and y ; here, x moves outside the membrane to the parent whereas y moves to the child existing inside the current membrane.

Computation process in a P system

Computation in a P system is performed in a non-deterministic and maximally parallel manner. The configuration changes from an initial state to the next state in an asynchronous manner, referred to as the transition of a P system. The computation is thus the continuous transition of the P system by applying the rules in a non-deterministic and maximally parallel manner until the system halts. The halting condition is achieved when no further rules can be applied and the output is obtained as the contents of the output membrane. A non-deterministic manner ensures that the rules are chosen at random. This randomness may lead to different transition paths. The order of application of rules is also an important aspect. Maximally parallel application of rules ensures that all possible rules are executed simultaneously in every transition step of the computation. The rules rewrite the multiset content inside a membrane. The execution of u → v rules is dependent mainly on the availability of the multiset composing the left side of the membrane u, which transforms into the right side multiset v.

Figure 4 shows a simple P System with four membranes 1, 2, 3, and 4. Membrane one is the outermost membrane that holds the output on halting. The system can transit through multiple process paths due to the non-deterministic nature of the computation.

FIGURE 4

FIGURE 4. P system.

Example steps of computation

• Step 1: Observing the initial configuration in membranes three and four Figure 4

m₃, R₃ = {a²b²}, {ab → ac, ac → c#} ab is assigned to rule ab → ac m₄, R₄ = {b²c²}, {bc → a, a → d_out} bc is assigned to rule bc → a.

• Step 2: After transition in step 1 we have

m₃, R₃ = {a²c²}, {ac → c#} ac is assigned to rule ac → c# # dissolves membrane three and multiset ac² moves into membrane two

m₄, R₄ = {a²}, {a → d_out} a is assigned to rule a → d_out d² is passed out of membrane four and no more transitions can happen in it.

• Step 3: After step 2 we have the membrane possible for transition

m₂, R₂ = {ac²d²}, {cd → e, e → e_out} cd is assigned to rule cd → e.

• Step 4:After step 3 membrane two new state

m₂, R₂ = {ae²}, {e → e_out} e is assigned by e → e_out e² is passed out to membrane one.

• membrane one is the output membrane so the computation halts here

m₁, R₁ = {e²}

The computation halts since no more rules can be executed.

Mutual Information

Mutual information measures the statistical dependency between two sets of data (here the image data sets) independent of the intensity values of images. The MI value between two images or voxels is maximum when the geometrical alignment between them is good. MI measures two sets of image data , A and B, obtained as follows:

$MI\left(A,B\right)=\sum _{a,b}\mathit{Prob}\left(a,b\right)\mathit{log}\frac{\mathit{Prob}\left(a,b\right)}{\mathit{Prob}\left(a\right)\mathit{Prob}\left(b\right)}.(4)$

Here, Prob (a, b) is the joint probability of a ϵ A and b ϵ B. Prob(a) and Prob(b) are the independent probabilities.

Methods

Tissue P system

Tissue P system (Pan and Perez-Jim´enez, 2010) can be viewed as the graph of P system membranes connected with bidirectional protein channels (Freund et al., 2005). The protein channels facilitate the communication (transportation) of multisets of objects between the membranes. The communication can be performed in a replicative manner where a copy of the multiset can be sent to all adjacent membranes attached to the communication channel (protein channel) or in a non-replicative manner where only one copy of the membrane is communicated to a particular membrane. Mathematically, the tissue P system (Bernardini and Gheorghe, 2005) of degree (number of membranes) n can be defined as

$\mathrm{\Gamma }=\left(O,{\mu }_1,{\mu }_2,\dots ,{\mu }_n,comm,{\mu }_{\mathit{out}}\right).(5)$

Here,

1. O is a finite set of objects (alphabets)

2. comm ⊆{1, 2, 3, .., n}×{1, 2, 3, … , n}

3. μ_out = {μ₁, μ₂, … , μ_n} is the output membrane

4. {μ₁, μ₂, … , μ_n} are the membranes of form μ₁ = {s_i,0, mul_i,0, Rule_i}

(a) s_i,0 is initial state of i^th membrane

(b) mul_i,0 multiset of i^th membrane

(c) Rule_i set of rules in i^th membrane

The tissue P system in Figure 5 is organized in the form of a multilevel membrane structure, with levels one and two having three membranes as child membranes. The output membrane or level 1 membrane is labeled usingμ₀. It contains three child membranes labeled by μ₀₁, μ₀₂, and μ₀₃ respectively; these form level two in the system. The level two membranes further contain three child membranes each. Child membranes are labelled as μ_ab, where a is the parent node and b is the child membrane. The system is interconnected with a bidirectional transportation channel that facilitates the transportation or communication of objects between the membranes. Each membrane contains multisets of objects along with the rules governing the evolution and communication of objects. The algorithm searches for the optimal solution among the floating image objects configured with the transform parameters inside the membranes. The objects in the solution space continuously evolve by utilizing the rules and are examined for the existence of a better optimized solution.

FIGURE 5

FIGURE 5. Tissue P system for TPSysIR algorithm.

The objects

The object in the solution space can be represented as the set of transformation parameters:

$Oj{b}_{rs}=x_{rs},y_{rs},{\theta }_{rs},Z_{rs}.(6)$

1. r = 1 … n and s = 1 … m

2. n is the membrane count of tissue P system

3. m is the object count in membrane r

4. θ_rs is the degree of rotation

5. Z_rs is the image scaling factor

6. x_rs is the displacement in x axis

7. y_rs is the displacement in y axis

The algorithm utilizes the MI metric as the optimization function to measure and maximize the similarity of the parameters in the evolved objects. The object evolves inside the membranes, and the local best is selected for each level 3 membrane. This local best value is communicated to the neighboring membrane and level 2 membranes through the bidirectional channel. The level 2 membranes communicate between themselves and their children to search for the global best solution. This solution is then transferred to the level 1 output membrane, thus representing the final solution.

The evolution rules

The evolution rules govern the evolution of float image object configurations inside the membrane; the evolution is performed so as to achieve the optimal best object according to the optimization criteria. Each level 3 membrane contains three optimal objects; $Ob{j}_{ab}^{t,best}$, the local optimal best object obtained at the t^th moment inside the ab^th membrane, $Ob{j}_{ab,n}^{t,best}$ , one of the optimal best object randomly selected among all the optimal objects received from the neighbors to the ab^th membrane, and $Ob{j}_{ab,u}^{t,best}$, the optimal best object transferred from the parent level 2 membrane u to ab^th child membrane.

The PSO (particle swarm optimization) technique is utilized here to govern the formation and execution of rule₁ and rule₂, and its velocity position equation is modified to define the evolution rules. These rules evolve the objects according to the position (configuration) and time.

$\begin{array}{ll}\hfill v_{k,ab}^{t+1}& ={\omega }^t{v}_{k,ab}^t+l_0{z}_0\left(Ob{j}_{ab}^{t,best}-Ob{j}_{k,ab}^t\right)+l_1{z}_1\left(Ob{j}_{ab,n}^{t,best}-Ob{j}_{k,ab}^t\right)\hfill \\ \hfill & +l_2{z}_2\left(Ob{j}_{ab,u}^{t,best}-Ob{j}_{k,ab}^t\right),\hfill \end{array}(7)$

$Ob{j}_{k,ab}^{t+1}=Ob{j}_{k,ab}^t+v_{k,ab}^{t+1},(8)$

$rul{e}_1\equiv {\left[Ob{j}_{k,ab}^t\right]}_{ab}^t\leftrightarrow {\left[Ob{j}_{k,ab}^{t+1}\right]}_{ab}^{t+1},(9)$

$\begin{array}{cc}\hfill Ob{j}_{ab}^{t+2,best}=& \mathit{max}\left\{\left\{MI\left(Ob{j}_{k,ab}^{t+1}\right)\right\}\cup MI\left(Ob{j}_{ab}^{t,best}\right)\right\}\hfill \end{array},(10)$

$rul{e}_2\equiv {\left[Ob{j}_{ab}^{t,best}\right]}_{ab}^{t+1}\leftrightarrow {\left[Ob{j}_{ab}^{t+2,best}\right]}_{ab}^{t+2}.(11)$

The velocity component is updated using (7) ω^t is the weight balancing factor which gradually decreases; l₀,l₁,and l₂ are the learning factor; z₀, z₁, and z₂ are the random numbers between 0 and 1. $Ob{j}_{k,ab}^t$ is the k^th float object in ab^th membrane having a configuration of the floating image. This is updated by rule₁ in Eqs. 8, 9 .This updates the new configuration of each object inside the membrane. The local best object of the membrane is selected by utilizing Eq 10, and the old one is replaced by the new local best utilizing rule₂ at Eq 11.

The communication rules

The communication (Paun and Paun, 2002) rules facilitate the transportation of objects between the membranes at inter- or intra-level utilizing the bidirectional channel connecting them.

Intra-level object communication rule

The local optimal object $Ob{j}_{ab}^{t,best}$ is updated during the evolution stage inside each membrane at level 3. These objects are further communicated to every neighboring membrane under the shared parent membrane. This process involves the creation of a duplicate copy of the local optimal object $Ob{j}_{ab}^{t,best}$ in each membrane, exchanging it with every other membrane under the common parent. The intracommunication rule is described as follows:

$rul{e}_3\equiv {\left[Ob{j}_{aj}^{t,best}\right]}_{ai}^t\leftrightarrow {\left[Ob{j}_{ai}^{t,best}\right]}_{aj}^t.(12)$

In Eq 12 rule₃ the objects $Ob{j}_{aj}^{t,best}$ and $Ob{j}_{ai}^{t,best}$ are the local optimal bests in membranes aj and ai, respectively. Both are located at level 3 under the same parent membrane. The objects are exchanged and rule₃ is executed in Eq 12.

Inter-level object communication rule

The copy of the local optimal object $Ob{j}_{ab}^{t,best}$ is updated during the evolution stage in each membrane at level 3, and this object is also communicated to the parent membrane at level 2. All the membranes in the child level 3 receive a copy of $Ob{j}_u^{t,best}$ optimal object updated at the parent level. This process involves simultaneous duplication and communication between a membrane at the parent and another at the child level. The intercommunication rule is described as

$rul{e}_4\equiv {\left[Ob{j}_u^{t,best}\right]}_{ab}^t\leftrightarrow {\left[Ob{j}_{ab}^{t,best}\right]}_u^t,(13)$

$rul{e}_5\equiv {\left[Ob{j}_u^{t,best}\right]}_u^t\leftrightarrow {\left[Ob{j}_u^{t,best}\right]}_0^t.(14)$

In (13), rule₄ exchanges the objects $Ob{j}_{ab}^{t,best}$ and $Ob{j}_u^{t,best}$ from membrane ab in level 3 and membrane u at level 2, respectively. The objects are first copied and are then exchanged between each child and parent membrane, executing rule₄ for each child’s membrane. The membrane at level 2 will have local optimal best objects from all its child membranes, and all its children will have a copy of the optimal object from its parent.

The (14) rule₅ communicates the object $Ob{j}_u^{t,best}$ optimal best of membrane u at level 2 is copied and sent to output membrane at level 1 as the global best.

The selection and substitution rules

After the inter level object communication stage, the membrane at level 2 has n copies of the object from each of its child nodes. Level 2 membrane selects the best among all the objects received from the child membranes and compares it with its local optimal best. The MI metric is utilized to perform the selection between the two best optimal values. The maximal object obtained from the above process is substituted as the current local best of this membrane. The rule can be described as

$\begin{array}{cc}\hfill Ob{j}_{ab}^{t+2,best}=& \mathit{max}\left\{\left\{MI\left(Ob{j}_{k,ab}^{t+1}\right)\right\}\cup MI\left(Ob{j}_{ab}^{t,best}\right)\right\}\hfill \end{array},(15)$

$rul{e}_6\equiv {\left[Ob{j}_u^{t,best}\right]}_u^t\to {\left[Ob{j}_u^{t+1,best}\right]}_u^{t+1}.(16)$

Eq. 15 examines the MI of all objects $Ob{j}_{k,ab}^{t+1}$ received from the child membranes along with the MI of the current best object $Ob{j}_u^{t,best}$ of membrane u for the maximum among all of them. This value is replaced as the new local best of membrane u by rule₆ in Eq 16.

The designed TPS utilizes the search capability of the PSO algorithm and explores the search space filled with floating image object solutions. The local object evolution is performed at the third level of the system; this generates local optimum objects. The optimized objects then move to the neighboring membrane and higher level 2 membranes.

The halting condition

The system is executed in the manner of steps; it is halted after the desired number of steps are performed. The optimal object obtained at the output level 1 membrane at that instance is recorded as the best solution to the problem.

The TPSysIR algorithm

The algorithm is designed using the TPS framework in the form of three membrane levels; the algorithm is shown in Figure 6. The system utilizes each level for specific evolution and optimization objectives. The level 3 membranes are utilized to evolve the floating image objects to achieve local optimal values at their level inside each membrane. Communication of local optimal objects among membranes under a common parent is done to optimize the configuration object further. This optimal object, obtained from each child membrane at level 3, is communicated to the level 2 membranes to form the global optimal solution. Level 2 membranes examine the optimality of the received objects from the child membranes to create a global maximal. The optimal value of level 2 is sent to the level 1 output membrane. The copy of the optimal value of the level 2 membrane is also sent back to the children’s membrane. The entire process, from the evolution of objects in the level 3 membrane to the transfer of global optima from level 2 to level 1, constitutes a step. The specified number of steps must be completed before the system halts; otherwise, the system restarts the evolution process utilizing the previously obtained locally optimized solutions.

FIGURE 6

FIGURE 6. TPSysIR algorithm.

Experimental setup

All the experiments were conducted using python (Python Programming Language, 2021) and plingua (P Lingua Programming Language, 2021) on a platform with Intel i5 four-core CPU at 2.5 GHz each and 8 gigabytes of RAM. The image data consisting of standard brain atlas was obtained from the Montreal Neurological Institute (MNI) (McConnell Brain Imaging Centre, 2021). The data was utilized by different optimization-based methods, utilizing MI as an optimization metric for image registration. The experiments are divided into two sets; first set of two experiments utilizes the single modal data and second set of three experiment utilizes the multimodal data.

Experiments with a single-modal image

1) Experiment 1: The float Figure 7B in this experiment is created by moving the original Figures 7A, 8 pixel units in x-axis up direction, six pixel units in y-axis in the left direction, and rotated 5° in a counterclockwise direction.

FIGURE 7

FIGURE 7. (A) Original Image: row first left. (B) Float Image Unscaled: row first right. (C) Float Image Scaled: second row center.

FIGURE 8

FIGURE 8. (A) Multimodal Experiment 1 T2: row first left. (B) Multimodal Experiment 1 T1: row first right. (C) Multimodal Experiment 2 T1: row second left. (D) Multimodal Experiment 1 T2: row second right. (E) Multimodal Experiment 3 T1: row third left. (F) Multimodal Experiment 1 T2: row third right.

2) Experiment 2: The original Figure 7A in this experiment is moved eight pixel-units in x-axis up direction, six pixel-units in y-axis, rotated 5° in a counterclockwise direction, and scaled into 0.8 times of its original size to create float Figure 7C.

Experiment with multimodal images

1) Experiment 1: The multimodal image set one contains images from two different modes, cerebrospinal fluid (CSF) section T1-weighted MRI Figure 8B having low signal and T2-weighted Figure 8A having high signal in the CSF section. T1 in X-Y plane image was utilized to create the float image Figure 9A after it was panned eight pixel units in x axis in the upward direction, six pixel units in Y axis in the left direction, rotated 5° in a counter clockwise direction, and scaled into 0.8 of its original size.

FIGURE 9

FIGURE 9. (A) Multi Modal Experiment 1 Float: row first left. (B) Multi Modal Experiment 1 GA output: row first right. (C) Multi Modal Experiment 1 PSO output: row second left. (D) Multi Modal Experiment 1 PSO and Powell Output: row second right. (E) Multi Modal Experiment 1 TPSysIR Output: row third center.

2) Experiment 2: The multimodal image set two contains T1 Figure 8C and T2 Figure 8D in the X-Z plane. The float Figure 10A was created from T2 after it was panned 10 pixel units in X axis in the upward direction, 13 pixel units in Z axis in the left direction, rotated 7° in a counter clockwise direction, and scaled into 1.1 of its original size. T1 is the target image.

FIGURE 10

FIGURE 10. (A) Multi Modal Experiment 2 Float: row first left. (B) Multi Modal Experiment 2 GA output: row first right. (C) Multi Modal Experiment 2 PSO output: row second left. (D) Multi Modal Experiment 2 PSO and Powell Output: row second right. (E) Multi Modal Experiment 2 TPSysIR Output: row third center.

3) Experiment 3: The multimodal image set two contains T1 and T2 images in the Y-Z plane. T1 Figure 8E is the target image while T2 Figure 8F was utilized as the float image. Figure 11A was created after it was panned 12 pixel units in Y axis in the upward direction, nine pixel units in Z axis in the left direction, rotated 10° in a counter clockwise direction, and scaled into 0.7 of its original size.

FIGURE 11

FIGURE 11. (A) Multi Modal Experiment 3 Float: row first left. (B) Multi Modal Experiment 3 GA output: row first right. (C) Multi Modal Experiment 3 PSO output: row second left. (D) Multi Modal Experiment 4 PSO and Powell Output: row second right. (E) Multi Modal Experiment 5 TPSysIR Output: row third center.

Experimental results and analysis

The experiments designed above are applied to the proposed TPSysIR algorithm along with three other optimization-based algorithms, GA, PSO, PSO, and POWELL. The experiments were repeated eight times for each method, and the maximum and minimum variance values were recorded and tabulated. MI value was also calculated for the output configuration on recorded data. The results for the variance along the X-axis and Y-axis and rotation angle(θ) are recorded in Table 1 for experiment 1, utilizing single modal images with no scaling factor. Table 2 shows that scaling factor variance S is considered for experiment 2 utilizing single modal image data with a scaling factor. Table 2 with multimodal experiments 1, 2, and 3 utilizes multimodal images for the experiments, and all four variables are considered.

TABLE 1

TABLE 1. Without scaling DATA.

TABLE 2

TABLE 2. With scaling DATA.

Table 1 results with no size scaling experiment show that PSO and POWELL have the lowest mean variance (Mean) and deviation(Δ) among the results in the Y-axis. TPSysIR has the lowest variance value on the X-axis and equals the deviation to PSO and POWELL. The mean rotation variance (θ) is the least in TPSysIR than all other algorithms including GA, PSO, and PSO and POWELL. TPSysIR has the maximum values, in mutual information, of 1.76 and the slightest deviation 0.02. The boxplot Figure 12 shows that the range of MI values obtained by PSO and Powell and TPSysIR is better than other algorithms, but TPSysIR obtains the range of least deviation. The maximum MI results for TPSysIR has shown 17.93%, 19.58%, and 9.61% improvement against the corresponding maximum MI values of GA, PSO, and PSO and POWELL algorithms.

FIGURE 12

FIGURE 12. Mutual Information ranges in each algorithm for Table 1 single modal experiment 1.

Table 2 single modal experiment 2 data includes the scaling factor along with all other parameters measured in experiment 1 to obtain results from all four methods. Notably in this experiment 2, the overall MI is lower than the previous experiment 1 as the similarity between the original and output image has decreased. The deviation in the X-axis is least in the TPSysIR algorithm. The deviation angle of rotation varies from 0.08 to 0.02 for GA to PSO and POWELL and TPSysIR. TPSysIR has a deviation value of 0.03 but has a lower max value than PSO and POWELL. The results of the experiments are shown in Figure 13B–E. PSO and POWELL and TPSysIR did a better job on image scaling. Mutual interference values show slightly better results for TPSysIR than PSO and POWELL, whereas other algorithms such as GA and PSO have lower scores. Angular bias is present in the output configuration images for GA Figure 13B and PSO Figure 13C. The boxplot Figure 14 shows that all algorithms gave minimum deviation in the range of MI values, but the maximum value range was obtained by TPSysIR to obtain best alignment. The maximum MI results for TPSysIR have shown 57.14%, 25%, and 7.84% improvement against the corresponding maximum MI values of GA, PSO, and PSO and POWELL algorithms, respectively.

FIGURE 13

FIGURE 13. (A) Single Modal Experiment 2 Float: row first left. (B) Single Modal Experiment 2 GA output: row first right. (C) Single Modal Experiment 2 PSO output: row second left. (D) Single Modal Experiment 2 PSO and Powell Output: row second right. (E) Single Modal Experiment 2 TPSysIR Output: row third center.

FIGURE 14

FIGURE 14. Mutual Information ranges in each algorithm for Table 2 single model experiment 2.

Table 2 multimodal experiment 1 data shows lower ranges of the MI values than in previous experiments, expressing further dissimilarity between the reference and the output image. The TPSysIR algorithm shows lower deviation values for X, Y, and rotation angle variance. The scalability factor deviation has relative range values for PSO and POWELL and TPSysIR, while MI values are best for TPSysIR. The boxplot Figure 15 shows that the ranges of MI values obtained by TPsysIR were the best. The maximum MI results for TPSysIR has shown 72.15%, 36%, and 21.42% improvement against corresponding maximum MI values of GA, PSO, and PSO and POWELL algorithms, respectively. The configuration results obtained for each method in the experiment for multimodal images are shown in Figure 9B–E.

FIGURE 15

FIGURE 15. Mutual Information ranges in each algorithm for Table 2 Multimodal Experiment 1.

Table 2 multimodal experiment 2 contains the data from the experiment performed over multimodal T1 and T2 images taken in the X-Z plane. The output in Figure 10B–E shows the output for the experiment. The table values for GA and PSO variance over X and Z have significantly higher values than PSO and Powell and TPSysIR. The MI value shows overlapping ranges for GA, PSO, and PSO and Powell while TPSysIR has higher ranges, as shown in the boxplot Figure 16. The TPSysIR has 29.52%, 24.72%, and 16.97% better MI values than GA, PSO, and PSO and Powell.

FIGURE 16

FIGURE 16. Mutual Information ranges in each algorithm for Table 2 Multimodal Experiment 2.

Table 2 multimodal experiment 3 data is obtained from the experiment performed over multimodal T1 and T2 images taken in the Y-Z plane. The output data for this experiment are shown in Figures 11B–E. The data shows the variance in Y of GA is the maximum of all other algorithms, and TPSysIR has the lowest. The MI range values of GA, PSO, and PSO and Powell show overlapping, while TPSysIR has a higher range of values. The same is confirmed in the boxplot in Figure 17. The TPSysIR MI values are 43.38%, 28.86%, and 17.83% are better than GA, PSO, and PSO and Powell respectively.

FIGURE 17

FIGURE 17. Mutual Information ranges in each algorithm for Table 2 Multimodal Experiment 3.

PSO and POWELL has good optimization capability due to its hybridization to acquire local and global search capabilities. The TPSysIR algorithm employs the TPS’s parallel execution capability, optimizing the local search and creating a globally optimal solution with faster convergence of results.

Conclusion

The method described in this paper utilizes the tissue P system’s parallel and simultaneous execution feature to guide its velocity position model-based rules. The novelty of this work is the use of the TPS, which enables faster convergence and high MI with the parallel feature using an optimization-based model to obtain parameters for image registration and the use of the PSO technique to make the evolution rules. The algorithm is tested on multimodal and unimodal MRI image sets to verify its effectiveness. The results of the tests prove it to be a good optimization-based solution to the image registration problem compared to other state-of-the-art algorithms.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found at: McConnell Brain Imaging Centre. https://www.bic.mni.mcgill.ca/Ser- vicesAtlases/ICBM152NLin2009.

Author contributions

SK—methodology, original draft, writing and editing, algorithm design, and implementation. SS—guidance, approach, correction, editing, and review.

Acknowledgments

The work here was tested utilizing images from The. nii files mni_icbm152_t1_tal_nlin_asym_09a.nii and mni_icbm152_t2_tal_nlin_asym_09a.nii obtained from the mni non-linear templates archive hosted at (Paun and Paun, 2002). Above usage required the following license copy (here the word “software” is utilized for the above files): copyright (C) 1993–2004 louis collins, mcconnell brain imaging center, montreal nerological institute, Mcgill University. Permission to use, Copy, Modify and DistributeI this Software and its Documentation for any Purpose and Without Fee is Here Granted, Provided That Above Copyright Notice Appear in all Copies the Authors and Mcgill University Make no Reperentations About The Suitability Of This Software For Any Purpose. It is Provided “as is” Without Express or Implied Warranty. The Authors are not Responsible for any Data Loss, Equipmenet Damage, Property Loss, or Injury To The Sunjects or Patients Resulting from Use or Misuse of this Software Package.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abdel-Basset, M., Fakhry, A. E., El-henawy, I., Qui, T., and Sangaiah, A. K. (2017). Feature and intensity based medical image registration using particle swarm optimization. J. Med. Syst. 41, 197. doi:10.1007/s10916-017-0846-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Ali, S., and Rittscher, J. (2019). Conv2Warp: An unsupervised deformable image registration with continuous convolution and Warping10. Avaliable from: https://arxiv.org/abs/1908.06194v1.

Google Scholar

Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., and Dalca, A. V. (2019). VoxelMorph: A learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38, 1788–1800. doi:10.1109/TMI.2019.2897538

CrossRef Full Text | Google Scholar

Bernardini, F., and Gheorghe, M. (2005). Cell communication in tissue P systems: Universality results. Soft Comput. 9, 640–649. doi:10.1007/s00500-004-0393-4

CrossRef Full Text | Google Scholar

Chel, H. D., and Nandi, D. (2013). “Image registration in noisy environment using particle swarm optimization,” in 2013 IEEE second international conference on image information processing (ICIIP-2013), 458–463. doi:10.1109/ICIIP.2013.6707634

CrossRef Full Text | Google Scholar

Freund, R., Paun, G., and Perez-Jimenez, M. J. (2005). Tissue P systems with channel states. Theor. Comput. Sci. 330 (1), 101–116. doi:10.1016/j.tcs.2004.09.013

CrossRef Full Text | Google Scholar

Gong, L., Zhang, C., Duan, L., Du, X., Liu, H., Chen, X., et al. (2019). Nonrigid image registration using spatially region-weighted correlation Ratio and GPU-acceleration. IEEE J. Biomed. Health Inf. 23 (2), 766–778. doi:10.1109/JBHI.2018.2836380

CrossRef Full Text | Google Scholar

Jiménez, M. J. P., and Núñez, A. R. (2005). Solving the Subset-Sum problem by P systems with active membranes. New Gener. comput. 23, 339–356. doi:10.1007/BF03037637

CrossRef Full Text | Google Scholar

Kakimoto, M., Endoh, Y., Shin, H., Ikeda, R., and Kusaka, H. (2019). Probabilistic solar irradiance forecasting by conditioning joint probability method and its application to electric power trading. IEEE Trans. Sustain. Energy 10 (2), 983–993. doi:10.1109/TSTE.2018.2858777

CrossRef Full Text | Google Scholar

Knops, Z. F., Maintz, J. B., Viergever, M. A., and Pluim, J. P. (2005). Normalized mutual information based registration using k-means clustering and shading correction. Med. Image Anal. 10 (3), 432–439. doi:10.1016/j.media.2005.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Lewis, J. P. (2001). Fast normalized cross-correlation. Quebec City: Canadian Image Processing and Pattern Recognition Society.

Google Scholar

Lin, C., Mimori, A., and Chen, Y. (2012). Hybrid particle swarm optimization and its application to multimodal 3D medical image registration. Comput. Intell. Neurosci. (2012), 561406. Article 6. doi:10.1155/2012/561406

PubMed Abstract | CrossRef Full Text | Google Scholar

Mansilla, L., Milone, D. H., and Ferrante, E. (2020). Learning deformable registration of medical images with anatomical constraints. Neural Netw., 269–279. doi:10.1016/j.neunet.2020.01.023

PubMed Abstract | CrossRef Full Text | Google Scholar

McConnell Brain Imaging Centre, (2021). Services atlases. Avaliable from: http://www.bic.mni.mcgill.ca/ServicesAtlases/ICBM152NLin2009 (Accessed Nov 16, 2021).

Google Scholar

P Lingua Programming Language, (2021). Lingua Programming Language. Avaliable from: http://www.p-lingua.org/wiki/index.php/Main_Page (Accessed Nov 23, 2021).

Google Scholar

Pan, L., and Perez-Jim´enez, M. J. (2010). Computational complexity ´of tissue-like P systems. J. Complex. 26 (3), 296–315. doi:10.1016/j.jco.2010.03.001

CrossRef Full Text | Google Scholar

Paun, A., and Paun, G. (2002). The power of communication: P systems with symport/antiport. New Gener. comput. 20, 295–305. doi:10.1007/BF03037362

CrossRef Full Text | Google Scholar

Paun, G. (2000). Computing with membranes. J. Comput. Syst. Sci. 61 (1), 108–143. doi:10.1006/jcss.1999.1693

CrossRef Full Text | Google Scholar

Paun, G. P. (2001). Systems with active membranes: Attacking NP complete problems. J. Automata, Lang. Comb. 6, 75–90.

Google Scholar

Prakash, J., Mandal, S., Razansky, D., and Ntziachristos, V. (2019). Maximum entropy based non-negative optoacoustic tomographic image reconstruction. IEEE Trans. Biomed. Eng. 66 (9), 2604–2616. doi:10.1109/TBME.2019.2892842

PubMed Abstract | CrossRef Full Text | Google Scholar

Python Programming Language, (2021). python. Avaliable from: https://www.python.org/ (Accessed Nov 16, 2021).

Google Scholar

Rajapakse, J. C., and Guojun, B. (1999). Functional MR image registration using a genetic algorithm, ICONIP’99. ANZIIS’99 ANNES’99 ACNN’99. 6th international conference on neural information processing. Proc. Cat. No.99EX378) 3, 922–927. doi:10.1109/ICONIP.1999.844660

CrossRef Full Text | Google Scholar

Ramamoorthy, S., Priyadharsini, K., and Rajaram, S. (2010). Intensity based image registration by maximization of mutual information. Int. J. Comput. Appl. 1, 1–7. doi:10.5120/432-637

CrossRef Full Text | Google Scholar

Rouet, J. M., Jacq, J. J., and Roux, C. (2000). Genetic algorithms for a robust 3-D MR-CT registration. IEEE Trans. Inf. Technol. Biomed. 4 (4), 126–136. doi:10.1109/4233.845205

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, G., Han, J., Zhao, Y., Wang, Z., and Du, H. (2017). A Review on medical image registration as an optimization problem. Curr. Med. Imaging Rev. 13 (3), 274–283. doi:10.2174/1573405612666160920123955

PubMed Abstract | CrossRef Full Text | Google Scholar

Susskind, J., Memisevic, R., Hinton, G., and Pollefeys, M. (2011). Modeling the joint density of two images under a variety of transformations. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2793–2800. doi:10.1109/CVPR.2011.5995541

CrossRef Full Text | Google Scholar

Wachowiak, M. P., Smol´ıkova, R., Zheng, Y., Zurada, J. M., and Elmaghraby, A. S. (2004). An Approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans. Evol. Comput. 8 (8), 289–301. doi:10.1109/tevc.2004.826068

CrossRef Full Text | Google Scholar

Wang, P., Wang, L., Leung, H., and Zhang, G. (2020). Super-resolution mapping based on spatial–spectral correlation for spectral imagery. IEEE Trans. Geosci. Remote Sens. 59 (3), 2256–2268. doi:10.1109/TGRS.2020.3004353

CrossRef Full Text | Google Scholar

Zhang, Z., Huo, P., Teng, J., Xue, W., and Suhuan, W. (2010). “Medical image registration algorithm with generalized mutual information and PSO-Powell hybrid algorithm,” in Advances in Swarm intelligence. Editors Y. Tan, Y. Shi, and K. C. Tan (Berlin Heidelberg: Springer), 160–166. doi:10.1007/978-3-642-13495-1_20

CrossRef Full Text | Google Scholar

Zhu, F., Gao, J., Yang, J., and Ye, N. (2022). Neighborhood linear discriminant analysis. Pattern Recognit. 123, 108422. doi:10.1016/j.patcog.2021.108422

CrossRef Full Text | Google Scholar

Zhu, F., Ning, Y., Chen, X., Zhao, Y., and Gang, Y. (2021). On removing potential redundant constraints for SVOR learning. Appl. Soft Comput. 102, 106941. doi:10.1016/j.asoc.2020.106941

CrossRef Full Text | Google Scholar

Zuo-zhu, L. (2007). Based on genetic algorithm mutual information of medical image registration. Artif. Intell. Identif. Technol. 1 (1), 1121–1122.

Google Scholar