{"id":18683505,"url":"https://github.com/narasimha1997/encoder-aas","last_synced_at":"2025-09-25T10:25:28.438Z","repository":{"id":104766804,"uuid":"287771028","full_name":"Narasimha1997/encoder-aas","owner":"Narasimha1997","description":"An architecture providing Universal Sentence Encoder as a service by exploiting job-level parallelism of multi-core architectures. The service can be used as a transformer model for downstream NLP and NLU tasks like Language modelling, Sentiment Classification, Question Answering, Semantic Search and more.","archived":false,"fork":false,"pushed_at":"2020-08-15T15:39:27.000Z","size":25730,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-27T23:43:44.702Z","etag":null,"topics":["c","clientserver","concurrency","cplusplus","load-balancer","natural-language-processing","natural-language-understanding","parallelism","python","python3","zmq"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Narasimha1997.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-15T15:15:15.000Z","updated_at":"2024-09-26T10:28:43.000Z","dependencies_parsed_at":"2023-05-29T17:30:39.321Z","dependency_job_id":null,"html_url":"https://github.com/Narasimha1997/encoder-aas","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fencoder-aas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fencoder-aas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fencoder-aas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fencoder-aas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Narasimha1997","download_url":"https://codeload.github.com/Narasimha1997/encoder-aas/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239535204,"owners_count":19654996,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","clientserver","concurrency","cplusplus","load-balancer","natural-language-processing","natural-language-understanding","parallelism","python","python3","zmq"],"created_at":"2024-11-07T10:14:48.890Z","updated_at":"2025-09-25T10:25:23.395Z","avatar_url":"https://github.com/Narasimha1997.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# encoder-aas\r\nAn architecture providing Universal Sentence Encoder as a service by exploiting job-level parallelism of multi-core architectures and can be used as a transformer model for downstream NLP and NLU tasks like Language modelling, Sentiment Classification, Question Answering, Semantic Search and more. Inspired from Bert-as-a-service.\r\n\r\n## Architecture:\r\n*encoder-aas* uses highly asynchronous programming model to exploit Job level parallelism of multi-core processor machines, \r\nsince it exploits job-level parallelism it can spin up highly concurrent and independant tensorflow session, each scheduled on a core.\r\nThe model we are using is Universal Sentence Encoder lite - more information can be found [here](https://www.tensorflow.org/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder_lite).\r\nThe model is very light weight and can run on a single processor, we expolited this opportunity to schedule multiple models on the same machine. The architecture is very simple and a developer can easily understand this.\r\n\r\n![architecture](https://github.com/Narasimha1997/encoder-aas/blob/master/images/use_encoder.jpg \"Architecture\")\r\n\r\n### Build the service:\r\nYou can use the dockerfile provided in `server/Dockerfile` to build the server. Follow the instructions below:\r\n1. Build the docker container:\r\n    ```\r\n    docker build . -t use_encoder\r\n    ```\r\n2. Run the docker container:\r\n    ```\r\n    docker run --rm -ti --net=host use_encoder --workers=4 --batch=1 --port=8000 --portpub=8001\r\n    ```\r\n    Parameters:\r\n      1. `workers` : Number of worker instances to run, use workers=number of cpus for max performance.\r\n      2. `batch` : Workers will consume from the queue only when entries=batch, `batch=1` is the default.\r\n      3. `port` : `ZMQSource` port\r\n      4. `portpub` : Service will publish results at this port.\r\n\r\n### Running the client:\r\nWe have provided a small client library that makes concurrent requests and and waits for responses. See `client` for its implementation.\r\nUsage example is shown here:\r\n```python\r\nfrom use_client import EncoderClient\r\nimport time\r\nresults = []\r\n\r\ndef callback(result) :\r\n\r\n    #print(result)\r\n    results.append(result)\r\n\r\n#create an encoder client\r\nencoder = EncoderClient(host = '127.0.0.1', port = 8000, port_pub = 8001, callback=callback)\r\n\r\n#start the receiver channel to receive results\r\nencoder.createChannel()\r\n\r\nsentences = ['Hello!', 'If you think you are bad, I am your dad!!', 'Just kiddnig', 'I am good']\r\n\r\nfor idx, sentence in enumerate(sentences)  :\r\n    encoder.generateEmbeddings(idx, sentence)\r\n\r\nwhile len(results) != len(sentences) :\r\n    time.sleep(0.01)\r\n\r\nprint(len(results))\r\n\r\nencoder.close()\r\n```\r\n\r\nIn case of any problems or feature requests, please feel free to raise an issue.\r\n\r\n### Acknowledgements:\r\n\r\n[Bert-as-a-Service](https://github.com/hanxiao/bert-as-service)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarasimha1997%2Fencoder-aas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnarasimha1997%2Fencoder-aas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarasimha1997%2Fencoder-aas/lists"}