{"id":22707104,"url":"https://github.com/cair/tm-xor-proof","last_synced_at":"2026-01-07T17:32:51.330Z","repository":{"id":44603418,"uuid":"312532149","full_name":"cair/TM-XOR-proof","owner":"cair","description":"#tsetlin-machine #machine-learning #game-theory #propositional-logic #pattern-recognition #bandit-learning #frequent-pattern-mining #learning-automata","archived":false,"fork":false,"pushed_at":"2022-08-24T13:32:19.000Z","size":70,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-02-04T21:45:10.207Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Cython","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cair.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-11-13T09:35:40.000Z","updated_at":"2022-08-27T12:36:41.000Z","dependencies_parsed_at":"2022-09-04T00:02:45.664Z","dependency_job_id":null,"html_url":"https://github.com/cair/TM-XOR-proof","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cair%2FTM-XOR-proof","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cair%2FTM-XOR-proof/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cair%2FTM-XOR-proof/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cair%2FTM-XOR-proof/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cair","download_url":"https://codeload.github.com/cair/TM-XOR-proof/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246243558,"owners_count":20746308,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-10T10:11:29.918Z","updated_at":"2026-01-07T17:32:51.292Z","avatar_url":"https://github.com/cair.png","language":"Cython","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Convergence-of-Tsetlin-Machine-for-Fundamental-Digital-Operations-the-XOR-Case\n\nThe Tsetlin Machine (TM) is a novel machine learning algorithm with several distinct properties, including transparent inference/learning and hardware-near building blocks. Although numerous papers explore the TM empirically, many of its properties have not yet been analyzed mathematically. In this article, we analyze the convergence of the TM when input is non-linearly related to output by the XOR-operator. Our analysis reveals that the TM, with just two conjunctive clauses, can converge surely to reproducing XOR, learning from training data over an infinite time horizon. Furthermore, the analysis shows how the hyper-parameter T guides clause construction so that the clauses capture the distinct sub-patterns in the data. Our analysis of convergence for XOR thus lays the foundation for analyzing other more complex logical expressions. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained state-of-the-art performance on several pattern recognition problems.\n\n(https://arxiv.org/abs/2101.02547)\n\n#### Code: XORDemo.py\n\nHere we have the basic experiment of the proof where we utilize a Tsetlin Machine having merely two clauses to learn the sub-patterns of only class 1. Towards the end of the code, the state of the Tsetlin automata at end of training in both the clauses are printed to reveal the pattern the each clause has learned. As prented below in the output, the clause 1 has learned the pattern (0 1) while the clause 2 learns pattern (1 0).\n\n```ruby\n#!/usr/bin/python\n# import cython\nimport numpy as np\nimport pyximport; pyximport.install(setup_args={\n                              \"include_dirs\":np.get_include()},\n                            reload_support=True)\n\nimport XOR\n\nX = np.random.randint(2, size=(10000,2), dtype=np.int32)\nY = np.ones([10000]).astype(dtype=np.int32)\n\nfor i in range(10000):\n    if X[i,0] == X[i,1]:\n        Y[i] = 0\n# Parameters for the Tsetlin Machine\nT = 1\ns = 3.9\nnumber_of_clauses = 2\nstates = 100 \nTh = 1\n\n\n# Parameters of the pattern recognition problem\nnumber_of_features = 2\n\n# Training configuration\nepochs = 200\n\n# Loading of training and test data\nNoOfTrainingSamples = len(X)*80//100\nNoOfTestingSamples = len(X) - NoOfTrainingSamples\n\n\nX_training = X[0:NoOfTrainingSamples,:] # Input features\ny_training = Y[0:NoOfTrainingSamples] # Target value\n\nX_test = X[NoOfTrainingSamples:NoOfTrainingSamples+NoOfTestingSamples,:] # Input features\ny_test = Y[NoOfTrainingSamples:NoOfTrainingSamples+NoOfTestingSamples] # Target value\n\n# This is a multiclass variant of the Tsetlin Machine, capable of distinguishing between multiple classes\ntsetlin_machine = XOR.TsetlinMachine(number_of_clauses, number_of_features, states, s, T, Th)\n\n# Training of the Tsetlin Machine in batch mode. The Tsetlin Machine can also be trained online\ntsetlin_machine.fit(X_training, y_training, y_training.shape[0], epochs=epochs)\n\n# Some performacne statistics\n\nprint (\"Accuracy on test data (no noise):\", tsetlin_machine.evaluate(X_test, y_test, y_test.shape[0]))\n\nfor clause in range(number_of_clauses):\n    print('Clause', clause+1),\n    for feature in range(number_of_features):\n        for tatype in range(2):\n            State = tsetlin_machine.get_state(clause,feature,tatype)\n            if State \u003e= 101:\n                Decision = 'In'\n            else:\n                Decision = 'Ex'\n            print('feature %d TA %d State %d' % (feature, tatype+1, State)),\n    print('/n')\n```\n#### Output\n```\nClause 1\nfeature 0 TA 1 State 99\nfeature 0 TA 2 State 101\nfeature 1 TA 1 State 101\nfeature 1 TA 2 State 12\n\nClause 2\nfeature 0 TA 1 State 101\nfeature 0 TA 2 State 70\nfeature 1 TA 1 State 100\nfeature 1 TA 2 State 101\n```\n\n#### Code: XORDemo_print.py\n\nIn this experiment, we are interested in seeing the distribution of the sub-potterns among clauses when different thresholds are used (``` T=2 ``` and ``` T=3 ```: a detail explanation can be found in the section four of the article). For this, the state of the Tsetlin Automata in each clause at each training epoch is saved into an excel file. After a socondary analysis on excel, plots in section four in the artical are created.\n\n```ruby\n#!/usr/bin/python\n\nimport numpy as np\nimport pyximport; pyximport.install(setup_args={\n                              \"include_dirs\":np.get_include()},\n                            reload_support=True)\n\nimport XOR_print\n\nsamples = 50\nX = np.random.random_integers(0,1,size=(samples,2)).astype(dtype=np.int32)\nY = np.ones([samples]).astype(dtype=np.int32)\n\nfor i in range(samples):\n    if X[i,0] == X[i,1]:\n        Y[i] = 0\n# Parameters for the Tsetlin Machine\nT = 2\ns = 3.9\nnumber_of_clauses = 5\nstates = 100 \nTh = 1\n\n\n# Parameters of the pattern recognition problem\nnumber_of_features = 2\n\n# Training configuration\nepochs = 200\n\n# Loading of training and test data\nNoOfTrainingSamples = len(X)*80//100\nNoOfTestingSamples = len(X) - NoOfTrainingSamples\n\n\nX_training = X[0:NoOfTrainingSamples,:] # Input features\ny_training = Y[0:NoOfTrainingSamples] # Target value\n\nX_test = X[NoOfTrainingSamples:NoOfTrainingSamples+NoOfTestingSamples,:] # Input features\ny_test = Y[NoOfTrainingSamples:NoOfTrainingSamples+NoOfTestingSamples] # Target value\n\n# This is a multiclass variant of the Tsetlin Machine, capable of distinguishing between multiple classes\ntsetlin_machine = XOR_print.TsetlinMachine(number_of_clauses, number_of_features, states, s, T, Th)\n\n# Training of the Tsetlin Machine in batch mode. The Tsetlin Machine can also be trained online\ntsetlin_machine.fit(X_training, y_training, y_training.shape[0], epochs=epochs)\n\n# Some performacne statistics\n\nprint (\"Accuracy on test data (no noise):\", tsetlin_machine.evaluate(X_test, y_test, y_test.shape[0]))\n```\n\n#### Output\n```\nAccuracy on test data (no noise): 1.0\n```\n\n#### Code: proof_of_the_convergence.py\n\nThis is the python implementation of Algorithm 1 in Apendix 1 of the ariticle. Mainly, the code generates the Transition Probability Matrix, computes the transpose\nof it and returns the result of power infinity. \n\n```ruby\nimport numpy as np\n\ns = 10\n\nDiaSum = np.zeros([256,1])\nM = np.zeros([256,256])\nMx = np.zeros([254,1,8])\nAbsorbingStates = np.zeros([2,1,8])\n#My = np.zeros([254,1,8])\nInput = np.zeros([4,2])\nInput[1,1],Input[2,0],Input[3,0],Input[3,1] = 1,1,1,1\n\nk = 0\nfor i in range(256):\n    if i == 105 or i == 150:\n        continue\n    k += 1\n    binary = \"{0:08b}\".format(i)\n    x = 0\n    #Mx[k-1,1,0] = i\n    for j in binary:\n        Mx[k-1,0,x] = int(j)\n        x += 1\n\ny = 0\nfor i in (105,150):\n    binary = \"{0:08b}\".format(i)\n    x = 0\n    #Mx[k-1,1,0] = i\n    for j in binary:\n        AbsorbingStates[y,0,x] = int(j)\n        x += 1\n    y += 1\n\nMx = np.vstack((AbsorbingStates,Mx))\nMy = Mx\n\ndef CalculateClasueOutputs(TADecisions, Input):   \n    TADecisions = np.reshape(TADecisions, (2, 4), order='F')\n         \n    Clause1 = 1\n    if (TADecisions[0,0] == 1 and Input[0] == 0) or (TADecisions[1,0] == 1 and Input[0] == 1) or (TADecisions[0,1] == 1 and Input[1] == 0) or (TADecisions[1,1] == 1 and Input[1] == 1):\n        Clause1 = 0\n        \n    Clause2 = 1\n    if (TADecisions[0,2] == 1 and Input[0] == 0) or (TADecisions[1,2] == 1 and Input[0] == 1) or (TADecisions[0,3] == 1 and Input[1] == 0) or (TADecisions[1,3] == 1 and Input[1] == 1):\n        Clause2 = 0\n        \n    return Clause1, Clause2\n\n\ndef ActivationProbability(FeedbackType, v):\n    if FeedbackType == 2:\n        if v == 0:\n            ActProbability = 0.5\n        else:\n            ActProbability = 1\n    else:\n        if v == 0:\n            ActProbability = 0.5\n        else:\n            ActProbability = 0.0\n            \n    return ActProbability \n            \n\ndef getLiteral(index, Input):\n    if index % 2 == 0:\n        literalType = 1\n    else: \n        literalType = 0\n\n    if index % 4 \u003c 2:\n        INx = 0\n    else:\n        INx = 1\n\n    if literalType == 1:\n        literal = Input[INx]\n    else:\n        literal = int(not Input[INx])\n        \n    return literal\n\ndef FeedbackProbability(FeedbackType, ClauseOutput, Literal, taDecision, change):\n    #print('FeedbackType', FeedbackType, 'ClauseOutput', ClauseOutput, 'Literal', Literal, 'taDecision', taDecision, 'change', change)\n    FeedProb = 0.0\n    Feedrew = 0.0\n    Feedinact = 0.0\n    if change == 1:\n        if FeedbackType == 1:\n            if ClauseOutput == 1:\n                if Literal == 1:\n                    if taDecision == 0:\n                        FeedProb = (s-1)/s\n    \n            else:\n                if Literal == 1:\n                    if taDecision == 1:\n                        FeedProb = 1/s\n    \n                else:\n                    if taDecision == 1:\n                        FeedProb = 1/s\n    \n        else:\n            if ClauseOutput == 1:\n                if Literal == 0:\n                    if taDecision == 0:\n                        FeedProb = 1\n    else:\n        if FeedbackType == 1:\n            if ClauseOutput == 1:\n                if Literal == 1:\n                    if taDecision == 1:\n                        Feedrew = (s-1)/s\n                        Feedinact = 1/s\n                    else: \n                        Feedinact = 1/s\n                else:\n                    if taDecision == 0:\n                        Feedrew = 1/s\n                        Feedinact = (s-1)/s\n            else:\n                if taDecision == 1:\n                    Feedinact = (s-1)/s\n                else:\n                    Feedrew = 1/s\n                    Feedinact = (s-1)/s\n        else:\n            if ClauseOutput == 1:\n                if Literal == 1:\n                    Feedinact = 1.0\n            else:\n                Feedinact = 1.0\n                        \n                \n                \n    if change == 1:                \n        FeedProb = FeedProb\n    else:\n        FeedProb = Feedrew + Feedinact\n        #print(FeedProb)  \n    return FeedProb\n\n\nfor xaxis in range(len(Mx)):\n    for yaxis in range(len(My)):\n        p = 0\n        for case in range(4): # different Inputs\n            if case == 0 or case == 3:\n                FeedbackType = 2\n            else:\n                FeedbackType = 1\n                \n            Clause1, Clause2 = CalculateClasueOutputs(Mx[xaxis,:,:],Input[case,:])\n            Clause1Prime, Clause2Prime = CalculateClasueOutputs(My[yaxis,:,:],Input[case,:])\n            ActProb = ActivationProbability(FeedbackType, Clause1+Clause2)\n            \n            if np.array_equal(Mx[xaxis,:,0:4], My[yaxis,:,0:4]):#clause1, no change case\n                FeedProb = 1\n                for index in range(4):\n                    literal = getLiteral(index, Input[case,:])\n                    change = 0\n                    FeedProb = FeedProb * FeedbackProbability(FeedbackType, Clause1, literal, Mx[xaxis,:,index], change)\n                \n                Pc1 = (ActProb * FeedProb) + (1 - ActProb)\n                \n            else:# clause1, change case\n                FeedProb = 1\n                for index in range(4):\n                    literal = getLiteral(index, Input[case,:])\n                    if Mx[xaxis,:,index] == My[yaxis,:,index]:\n                        change = 0\n                    else: \n                        change = 1\n                    \n                    FeedProb = FeedProb * FeedbackProbability(FeedbackType, Clause1, literal, Mx[xaxis,:,index], change)\n                \n                Pc1 = ActProb * FeedProb\n                                \n            if np.array_equal(Mx[xaxis,:,4:8], My[yaxis,:,4:8]):#clause2, no change case\n                FeedProb = 1\n                for kk in range(4):\n                    index = kk+4\n                    literal = getLiteral(index, Input[case,:])\n                    change = 0\n                    FeedProb = FeedProb * FeedbackProbability(FeedbackType, Clause2, literal, Mx[xaxis,:,index], change)\n                \n                Pc2 = (ActProb * FeedProb) + (1 - ActProb)\n                                \n            else:#clause2, change case\n                FeedProb = 1\n                for kk in range(4):\n                    index = kk+4\n                    literal = getLiteral(index, Input[case,:])\n                    if Mx[xaxis,:,index] == My[yaxis,:,index]:\n                        change = 0\n                    else: \n                        change = 1\n                    \n                    FeedProb = FeedProb * FeedbackProbability(FeedbackType, Clause2, literal, Mx[xaxis,:,index], change)\n                \n                Pc2 = ActProb * FeedProb\n                \n                \n            p += 0.25 * Pc1 * Pc2\n            #print(Pc1, Pc2)\n\n        M[yaxis, xaxis] = p\n        \n\nMM = M.transpose() \n        \nfor j in range(256):\n    DiaSum[j,0] = sum(MM[j,:]) # sum of all row\n    \nnp.savetxt('transitionmatrixstandard.csv', MM)\n            \n\nmm=np.linalg.matrix_power(MM, 1000000)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcair%2Ftm-xor-proof","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcair%2Ftm-xor-proof","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcair%2Ftm-xor-proof/lists"}