{"id":22462369,"url":"https://github.com/adeboyed/azure_serverless_blog","last_synced_at":"2026-05-09T00:37:32.871Z","repository":{"id":76833379,"uuid":"114012393","full_name":"adeboyed/azure_serverless_blog","owner":"adeboyed","description":"Repo + Resources for blog wrriten for Microsoft about Serverless","archived":false,"fork":false,"pushed_at":"2017-12-14T14:09:36.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-01T18:35:44.542Z","etag":null,"topics":["azure","azure-functions","blog-wrriten","java","javascript","microsoft","serverless","service-bus-queue"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adeboyed.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-12T16:19:44.000Z","updated_at":"2017-12-14T14:10:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"c766c84f-2bc3-4a05-8ee0-6281f21de021","html_url":"https://github.com/adeboyed/azure_serverless_blog","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adeboyed%2Fazure_serverless_blog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adeboyed%2Fazure_serverless_blog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adeboyed%2Fazure_serverless_blog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adeboyed%2Fazure_serverless_blog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adeboyed","download_url":"https://codeload.github.com/adeboyed/azure_serverless_blog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245858889,"owners_count":20684062,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","azure-functions","blog-wrriten","java","javascript","microsoft","serverless","service-bus-queue"],"created_at":"2024-12-06T09:09:16.958Z","updated_at":"2026-05-09T00:37:27.840Z","avatar_url":"https://github.com/adeboyed.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serverless + Service Bus Queue = Magic\n\nhttps://blogs.msdn.microsoft.com/uk_faculty_connection/2017/12/14/serverless-service-bus-queue-magic/\n\n## About Me\n\nHello! My name is David Adeboye and I am currently studying Computer Science at University of Cambridge. My main interest is in site reliability and concurrent and distributed systems, and in particular, I like designing system architectures in interesting ways. LinkedIn profile: https://www.linkedin.com/in/david-adeboye/ \n\n## Introduction\n\nOne of the ideas in distributed systems is the producer-consumer model, in this blog we'll look into using Azure functions to process items on an Azure Service Bus queue. The advantages of this is that it means you are not having to bear the costs of a running a virtual machine all the time, especially if your workload can be quite infrequent. Another use is when your workload is highly parallelisable, meaning you can run several instances of your code on different data.\n\nWe will partially follow the step by step tutorial provided and show how to an example of functions by creating a twitter bot using Markov models. All the source for this blog can be found on my github. (Link found at the bottom)\n\n## Prerequisite\n\nI do use both Java in and Javascript in this project, this is both to demonstrate how the Service Bus queue supports multiple different languages and because Java is still in preview for Azure functions. The code is well-commented and should be easy to follow even if you've never used either language before.\n\nTheory about Hidden Markov Models would be beneficial but is not required, for brevity I use a library to generate the words, however better results may be achieved by using a custom solution.\n\nYou will require an Azure account, sign up here for free: http://portal.azure.com/\n\nYou will also require Twitter API credentials, obtainable from here: https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens Please generate an access token + secret for yourself.\n\n## End Product\n\nYou should build a bot that will wait for people to tweet on a particular hashtag with a word, and then tweet back at them with song lyrics generated using a hidden markov model and a starting word.\n\n## Implementation\n\nSince we are using the Service Bus queue, we will require 2 things, producer(s) and consumer(s). \n\n### Producer\n\nThis was implemented using Java, for now I ran it on my local computer, but it could easily run on an Azure Virtual Machine.\n\nThe first step is to set it up as a maven project, this is so that we can include the required dependencies easily, essentially we need the Azure service bus library and I'm using twitter4j to communicate with the Twitter API.\n\n```\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cproject xmlns=\"http://maven.apache.org/POM/4.0.0\"\n         xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n         xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\"\u003e\n    \u003cmodelVersion\u003e4.0.0\u003c/modelVersion\u003e\n\n    \u003cgroupId\u003eTwitterMSPBot\u003c/groupId\u003e\n    \u003cartifactId\u003etwitter-msp-bot\u003c/artifactId\u003e\n    \u003cversion\u003e1.0-SNAPSHOT\u003c/version\u003e\n\n    \u003cdependencies\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003ecom.microsoft.azure\u003c/groupId\u003e\n            \u003cartifactId\u003eazure-servicebus\u003c/artifactId\u003e\n            \u003cversion\u003e0.9.8\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.twitter4j\u003c/groupId\u003e\n            \u003cartifactId\u003etwitter4j-core\u003c/artifactId\u003e\n            \u003cversion\u003e[4.0,)\u003c/version\u003e\n        \u003c/dependency\u003e\n    \u003c/dependencies\u003e\n\n\u003c/project\u003e\n```\n\nNext we'll need to create a service bus, head to portal.azure.com, login then click `New` and finally type `Service Bus`.![](photos/service_bus_search.png)\n\nClick `Create` and fill in the information, **taking note of your namespace.** for the rest of this tutorial we shall take the namespace to be *twitter-msp-bus*\n\nOnce it has been deployed, head to the settings page and click on `Shared access policies \u003e RootManageSharedAccessKey ` you'll need to **take note of the primary key shown.**\n\n![](photos/shared_policies.png)\n\n\n\nFinally, we need to create a queue, click `Queue` then `+ Queue` at the top. The settings don't need to be changed, but just create a queue and take note of the name.\n\n![](photos/add_queue.png)\n\nNext create a new class, I've named mine `Producer.java` Create a main function.\n\nThe first thing we'll need is to communicate with the Azure Service Bus. Copy and paste the code with your namespace and primary key in the relevant places:\n\n```Java\nConfiguration config =\n                ServiceBusConfiguration.configureWithSASAuthentication(\n                        \"twitter-msp-bus\",\n                        \"RootManageSharedAccessKey\",\n                        \"[primary key]\",\n                        \".servicebus.windows.net\"\n                );\n```\n\nNext we'll need to get the Twitter functionality, make sure to enter your own twitter credentials.\n\n```JAva\nTwitterFactory factory = new TwitterFactory();\nAccessToken accessToken = new AccessToken( [twitter_access_token], [twitter_access_secret] );\nTwitter twitter = factory.getInstance();\n\ntwitter.setOAuthConsumer( [twitter_consumer_key] ) , [twitter_consumer_secret] );\ntwitter.setOAuthAccessToken(accessToken);\n```\n\nNext we shall initialise the service bus connection and create a query to search on Twitter, in this example we are searching for the hashtag **#TwitterMSPBot**\n\n```\nServiceBusContract service = ServiceBusService.create(config);\n\nQuery query = new Query(\"#TwitterMSPBot\");\nQueryResult queryResult;\n```\n\nThe next part will need to be srrounded in a try and catch because it can throw errors.\n\nThe part below will indefinetly look on twitter for the latest tweets with the hashtag and add them to the service bus message queue in the JSON format. It will also remove any unnecessary details such as other user's usernames and any hashtags, including the one we are searching for.\n\n```Java\nwhile(true){\n\tSystem.out.println(\"Started Loop\");\n\tqueryResult  = twitter.search( query );\n\tList\u003cStatus\u003e statuses = queryResult.getTweets();\n  \tif ( statuses.size() \u003e 0 ){\n    \tfor ( Status status : statuses ){\n      \t\tif ( !seenTweets.contains( status.getId() ) ){\n            \tseenTweets.add( status.getId() );\n              \tSystem.out.println(\"Found Tweet!\");\n              \t//Put this on the queue\n              \ttry {\n                \tString text = status.getText().trim();\n\n                  \t//Remove any @s\n                  \ttext = text.replaceAll(\"@[a-z|1-9|A-Z]+\", \"\").trim();\n\n                  \t//Remove any hashtags\n                  \ttext = text.replaceAll(\"#[a-z|A-Z|1-9]+\", \"\").trim();\n                  \ttext = text.toLowerCase();\n\n                  \tJSONObject jsonObject = new JSONObject();\n                  \tjsonObject.put(\"author\", status.getUser().getScreenName() );\n                  \tjsonObject.put(\"tweet\", text );\n                  \tBrokeredMessage message = new BrokeredMessage( jsonObject.toString() );\n                  \tSystem.out.println(\"Adding to the queue: \" + jsonObject.toString() );\n                  \tservice.sendQueueMessage( queueName, message );\n              \t} catch ( JSONException e ){\n                  System.out.println(\"Could not add tweet to messaging queue due to JSON parsing\");\n        \t\t}\n\t\t\t}\n        }\n  }\n  Thread.sleep(60000); //Sleep for a minute before checking again\n```\n\nAnd then we are done!\n\n### Consumer\n\nThis was implemented using Javascript, with a bit of nodejs.\n\nReturning to the Azure portal as we will be creating the serverless functions, click `New` and type `Function App`\n\n![](photos/function.png)\n\nCreate an app with Windows as OS, in this case we shall name *twitter-bot-msp* and I will use it this for the rest of the walkthrough.\n\nOnce it has been deployed, we will need to create the function, you want a 'Queue Trigger' function.\n\n![](photos/queue_trigger.png)\n\nSelect the language as Javascript, and name your function, I've named it 'queue-trigger' but you can ignore everything else, we'll connect it to the service bus later.\n\n####  Creating the function\n\n1. Navigate to `https://twitter-bot-msp.scm.azurewebsites.net/`  and click `Debug Console \u003e CMD`\n\n2. Type in `cd site\\wwwroot` to get to the site's home directory then type `npm init` to start an npm package. Enter data for the relevant fields. Here is mine below:\n\n   ![](photos/npm_init.png)\n\n3. Type `npm install markovchain` to install the hmm module, then type `npm install twitter` to install the twitter module\n\n4. Return back to the Azure portal and click on the file named 'function.json', you want to make sure that the type, the queueName, and the connection are changed to the correct parameters as shown below\n\n   ```JSON\n   {\n     \"bindings\": [\n       {\n         \"name\": \"myQueueItem\",\n         \"type\": \"serviceBusTrigger\",\n         \"direction\": \"in\",\n         \"queueName\": \"twitter-queue\",\n         \"connection\": \"twitter-bot-msp_RootManageSharedAccessKey_SERVICEBUS\",\n         \"accessRights\": \"Manage\"\n       }\n     ],\n     \"disabled\": false\n   }\n   ```\n\n   ​\n\n5. Return back to index.js and copy in the following code, you'll need to enter your Twitter credientials again below in the correct position. In the corpus you'll need to paste in text for your markov model to train from. In my case I entered lyrics from one of most-listened artists this year, according to Spotify. (Omittted for copyright reasons). I've explained what each part does below:\n\n   ```javascript\n   var Twitter = require('twitter');\n   var MarkovChain = require('markovchain');\n\n   const corpus = ``;\n\n   module.exports = function (context, myQueueItem) {\n   \tvar tweetLength = Math.floor((Math.random() * 5) + 5);\n       const starting_word = myQueueItem.tweet;\n\n       var client = new Twitter({\n       \tconsumer_key: '[twitter_access_consumer_key]',\n         \tconsumer_secret: '[twitter_consumer_secret]',\n         \taccess_token_key: '[twitter_access_token]',\n         \taccess_token_secret: '[twitter_access_secret]'\n       });\n     \n     \tconst lyricGen = quotes = new MarkovChain(corpus);\n     \tconst sentence = quotes.start( starting_word ).end( tweetLength ).process();\n     \tconst toTweet = \"@\" + myQueueItem.author + \" \" + sentence;\n     \tcontext.log('Generated sentence to tweet: ' + sentence);\n\n     \tclient.post('statuses/update', {status: toTweet},function(error, tweet, response) {\n       \tif(error) throw error;\n         \t\t//context.log(tweet);  // Tweet body. \n               //context.log(response);  // Raw response object. \n       });\n       context.done();\n   };\n   ```\n\n\n\n\n```javascript\nvar Twitter = require('twitter');\nvar MarkovChain = require('markovchain');\n\nconst corpus = ``;\n```\n\nThis piece of code imports both the Twitter and Markov chain library, for brevity I am using a library instead of implementing my own. The corpus variable should be where you paste in some text for your Markov model, I used about 1000 lines of lyrics. Basically the more the better.\n\n```javascript\nvar tweetLength = Math.floor((Math.random() * 5) + 5);\nconst starting_word = myQueueItem.tweet;\n```\n\nTo make things more interesting we want to vary the length of the tweets, tweetLength should be a random number between 5 and 10, denoting the number of words we are going to generate. The starting_word should be taken from the tweet as the start of the markov model.\n\n```javascript\nvar client = new Twitter({\n    consumer_key: '[twitter_access_consumer_key]',\n    consumer_secret: '[twitter_consumer_secret]',\n    access_token_key: '[twitter_access_token]',\n    access_token_secret: '[twitter_access_secret]'\n});\n```\n\nThis just creates and authenicates the Twitter object we are going to use for tweeting\n\n```javascript\nconst lyricGen = quotes = new MarkovChain(corpus);\nconst sentence = quotes.start( starting_word ).end( tweetLength ).process();\nconst toTweet = \"@\" + myQueueItem.author + \" \" + sentence;\ncontext.log('Generated sentence to tweet: ' + sentence);\n```\n\nFirstly we generate our markov model and store it in an object named `lyricGen`, then we use this to generate a new lyric using the starting word we got from the original tweet. Then finally we draft a tweet to send, and we log the sentence we generate. In a future iteration we should train once and just store the probabilities in a file to be loaded up everytime this function runs.\n\n```javascript\nclient.post('statuses/update', {status: toTweet},function(error, tweet, response) {\n    if(error) throw error;\n    //context.log(tweet);  // Tweet body. \n    //context.log(response);  // Raw response object. \n});\ncontext.done();\n```\n\nThis piece of code will tweet the sentence from your account. In testing you should uncomment those lines in case there are any errors, especially the raw response object. Context.done tells the function that we have completeted execution and we can be terminated, hopefully that tweet posted!\n\n**And then we are done, we have succesfully created a twitter bot using Azure Service bus and serverless.**\n\n### In Action \n\n#### Producer running:\n\n![](photos/run_producer.png)\n\n#### Consumer runs:\n\n![](photos/run_consumer.png)\n\n#### Example tweet produced:\n\n![](photos/run_tweet.png)\n\n### Closing\n\nThere are several improvements that can be made to this system, however the general idea was demonstrated. This could be used in applications such as batch processing where you may not particularly want a dedicated virtual machine running all the time, but you want it there when you need them. \n\nAnother use case could be the ability to test your application as part of your deployment, instead of having a dedicated machine for testing, or running it locally, you could have an Azure function running to test your application when you need it!\n\nExtra points for guessing the artist!\n\n\n## Resources for reference and further reading\n\n1. Source code: https://github.com/adeboyed/azure_serverless_blog\n2. Azure Service bus: https://azure.microsoft.com/en-gb/services/service-bus/\n3. Introducing Azure functions: https://azure.microsoft.com/en-gb/blog/introducing-azure-functions/\n4. npm reference for the markov models: https://www.npmjs.com/package/markov-chains-text\n5. npm reference for the twitter api: https://www.npmjs.com/package/twitter\n6. java reference for the twitter api: http://twitter4j.org/en/\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadeboyed%2Fazure_serverless_blog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadeboyed%2Fazure_serverless_blog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadeboyed%2Fazure_serverless_blog/lists"}