{"id":16276532,"url":"https://github.com/virgesmith/hedfq","last_synced_at":"2025-04-08T16:53:03.325Z","repository":{"id":87710789,"uuid":"328968624","full_name":"virgesmith/hedfq","owner":"virgesmith","description":"Homomorphic encryption client-server demo","archived":false,"fork":false,"pushed_at":"2021-01-12T18:22:20.000Z","size":26,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-14T13:28:25.604Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/virgesmith.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-12T11:41:42.000Z","updated_at":"2021-01-12T18:27:15.000Z","dependencies_parsed_at":"2023-03-13T18:39:50.652Z","dependency_job_id":null,"html_url":"https://github.com/virgesmith/hedfq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/virgesmith%2Fhedfq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/virgesmith%2Fhedfq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/virgesmith%2Fhedfq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/virgesmith%2Fhedfq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/virgesmith","download_url":"https://codeload.github.com/virgesmith/hedfq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247888243,"owners_count":21012996,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T18:48:44.701Z","updated_at":"2025-04-08T16:53:03.292Z","avatar_url":"https://github.com/virgesmith.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hedfq: Homomorphically Encrypted DataFrame Query\n\nHomomorphic encryption permits mathematical operations to be executed directly on encrypted data, yielding an encrypted result that, when decrypted, is the same as you would get on the unencrypted input.\n\nThis means that, for example, you can securely store encrypted data on a server and perform queries on it without ever having to worry about server-side encryption keys being compromised - they simply aren't there. The recipient of the data must decrypt the result.\n\nThis demo uses the [pyfhel](https://github.com/ibarrond/Pyfhel) package, which provides python bindings to a version of the [SEAL](https://www.microsoft.com/en-us/research/project/microsoft-seal/) library.\n\n## Simple test\n\n```\npython test_df.py\n```\n\n## Client-server demo\n\nFor the purposes of the demonstration we create a server that stores encrypted data and will aggregate it (i.e. perform addition)  on request *without requiring decryption*.\n\nThe server permits a single encrypted dataset to be uploaded (along with a corresponding public key and a encryption parameters, **but not** the private key that would permit decryption).\n\nThen, the encrypted data can be downloaded directly, aggregrated, or deleted entirely. Only the client can actually decrypt the data.\n\nThe client script does the following:\n\n- loads an unencrypted dataset\n- encrypts the data and uploads it to the server\n- asks the server for aggregated data and decrypts the result (x3)\n- deletes the data from the server\n\nFirst start the server:\n\n```bash\nFLASK_APP=server flask run\n```\n\nThen run the client script:\n\n```bash\npython client.py\n```\n\nYou should get something like this:\n\n```text\nget before dataset has been uploaded:\n400 b'no encrypted data registered'\nupload encrypted dataset:\n200 b'152 rows'\ncheck dataset ok:\naggregate by area and decrypt:\n                  2018     2019     2020     2021     2022\nAGE GROUP SEX\n0-4       F    13710.0  13474.8  13230.4  13051.3  12772.1\n          M    14509.0  14111.1  13844.6  13536.2  13285.6\n10-14     F    13896.0  14218.2  14570.1  14788.2  14993.0\n          M    14461.0  14606.9  14940.6  15092.5  15431.4\n15-19     F    12772.0  12458.8  12423.2  12861.1  13353.1\n          M    13537.0  13594.8  13633.1  13933.5  14205.8\n20-24     F    14168.0  13875.8  13616.7  13312.6  12859.5\n          M    15684.0  15176.3  14782.1  14343.7  13970.2\n25-29     F    15630.0  15629.3  15465.7  15119.7  14690.6\n          M    15869.0  15933.1  15922.4  15799.0  15441.3\n30-34     F    15534.0  15618.7  15588.0  15629.5  15878.5\n          M    14547.0  14775.0  14942.4  15080.3  15331.5\n35-39     F    14381.0  14614.9  14776.0  14955.3  15041.5\n          M    13172.0  13374.1  13465.3  13589.3  13831.6\n40-44     F    13038.0  13004.5  13282.7  13541.8  14038.1\n          M    12306.0  12284.8  12404.8  12665.4  12956.3\n45-49     F    15882.0  15443.6  14790.9  14092.4  13410.2\n          M    14772.0  14269.7  13724.7  13117.1  12524.7\n5-9       F    14921.0  14962.6  14898.4  14533.8  14305.5\n          M    15551.0  15731.7  15619.3  15503.5  15194.0\n50-54     F    17548.0  17203.3  16881.7  16586.1  16205.3\n          M    16393.0  16123.8  15636.1  15253.1  14922.5\n55-59     F    17069.0  17256.5  17574.7  17779.4  17627.3\n          M    16232.0  16317.9  16706.6  16848.5  16609.0\n60-64     F    14779.0  15314.1  15695.4  16117.8  16705.2\n          M    13961.0  14321.9  14661.0  14963.9  15378.7\n65-69     F    13623.0  13480.6  13453.5  13691.9  13998.4\n          M    12565.0  12455.5  12493.8  12663.4  12956.3\n70-74     F    12655.0  13126.8  13477.3  13504.4  12949.8\n          M    11912.0  12268.8  12282.4  12348.4  11788.1\n75-79     F     9132.0   9222.9   9340.3   9844.3  10748.4\n          M     7893.0   8068.3   8478.4   8876.8   9758.4\n80-84     F     7582.0   7773.3   7842.3   7723.6   7660.1\n          M     5704.0   5946.4   5989.5   6000.4   6053.8\n85-89     F     4671.0   4727.8   4816.1   4872.2   5047.5\n          M     2986.0   3055.0   3137.6   3232.1   3344.0\n90+       F     2784.0   2839.6   2867.8   2904.9   2924.6\n          M     1242.0   1280.0   1332.8   1380.3   1432.0\naggregate by age and decrypt:\n                             2018     2019     2020     2021     2022\nAREA                 SEX\nDarlington           F    54647.0  54721.9  54790.3  54854.8  54911.3\n                     M    51919.0  51973.2  52037.8  52078.6  52112.2\nHartlepool           F    47718.0  47796.4  47863.8  47925.7  47975.2\n                     M    45524.0  45569.8  45594.7  45624.0  45645.8\nMiddlesbrough        F    71109.0  71113.0  71063.0  71019.1  70980.7\n                     M    69436.0  69418.2  69359.8  69281.6  69195.2\nRedcar and Cleveland F    70301.0  70614.8  70874.1  71110.7  71341.5\n                     M    66417.0  66733.9  67005.2  67243.2  67462.0\naggregate by sex and decrypt:\n                                  2018    2019    2020    2021    2022\nAREA                 AGE GROUP\nDarlington           0-4        5917.0  5746.8  5615.3  5475.0  5410.3\n                     10-14      6410.0  6508.2  6654.0  6645.2  6760.0\n                     15-19      5656.0  5570.0  5601.6  5808.7  5936.3\n                     20-24      5429.0  5345.8  5190.0  5019.3  4894.7\n                     25-29      6267.0  6313.5  6304.3  6159.8  6026.2\n...                                ...     ...     ...     ...     ...\nRedcar and Cleveland 70-74      8333.0  8552.3  8598.7  8534.2  8059.4\n                     75-79      5848.0  5998.0  6185.4  6517.7  7081.1\n                     80-84      4228.0  4439.8  4563.6  4599.3  4656.5\n                     85-89      2389.0  2461.8  2512.4  2578.1  2712.4\n                     90+        1295.0  1320.8  1365.1  1394.3  1425.0\n\n[76 rows x 5 columns]\naggregate with invalid param:\nb\"invalid or missing aggregation parameter: INVALID. must be one of ['AREA', 'AGE GROUP', 'SEX']\"\ndelete encrypted data on server:\ncheck encrypted data no longer on server:\n```\n\n## Caveats\n\n- Each (numeric) value is encrypted individually, rather than the whole dataset. Non-numeric values are not encrypted.\n- The encrypted (serialised) data is *much* larger than the original: 32MB vs 8.1kB.\n\n## Resources\n\nhttps://en.wikipedia.org/wiki/Homomorphic_encryption\n\nhttps://pyfhel.readthedocs.io/en/latest/\n\nhttps://blog.openmined.org/build-an-homomorphic-encryption-scheme-from-scratch-with-python/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvirgesmith%2Fhedfq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvirgesmith%2Fhedfq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvirgesmith%2Fhedfq/lists"}