{"id":13639406,"url":"https://github.com/aaliraaza/MerakiAnalytics","last_synced_at":"2025-04-19T22:32:30.022Z","repository":{"id":186120975,"uuid":"100622145","full_name":"aaliraaza/MerakiAnalytics","owner":"aaliraaza","description":"Azure PaaS Implementation using Lambda Architecture of Cisco Meraki In-Store Location Analytics","archived":false,"fork":false,"pushed_at":"2017-08-23T12:34:06.000Z","size":131367,"stargazers_count":6,"open_issues_count":0,"forks_count":9,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-08-03T01:14:43.932Z","etag":null,"topics":["arm","azuresqldb","eventhubs","powerbi","streamanalytics","web"],"latest_commit_sha":null,"homepage":"","language":"PowerShell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aaliraaza.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-08-17T16:06:36.000Z","updated_at":"2023-04-09T11:13:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"775cbd36-d848-471c-aba6-801fff94fb92","html_url":"https://github.com/aaliraaza/MerakiAnalytics","commit_stats":null,"previous_names":["aaliraaza/merakianalytics"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaliraaza%2FMerakiAnalytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaliraaza%2FMerakiAnalytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaliraaza%2FMerakiAnalytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaliraaza%2FMerakiAnalytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aaliraaza","download_url":"https://codeload.github.com/aaliraaza/MerakiAnalytics/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223810389,"owners_count":17206756,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arm","azuresqldb","eventhubs","powerbi","streamanalytics","web"],"created_at":"2024-08-02T01:01:00.320Z","updated_at":"2024-11-09T09:30:57.316Z","avatar_url":"https://github.com/aaliraaza.png","language":"PowerShell","funding_links":[],"categories":["Administration"],"sub_categories":["Contents"],"readme":"# Cisco Meraki Location Analytics\nAzure PaaS Implementation using Lambda Architecture of Cisco Meraki In-Store Location Analytics\n\n## **What is Cisco Meraki?**\nhttps://meraki.cisco.com/solutions/location-analytics\n\nCisco Meraki Location Analytics displays real-time location statistics to improve customer engagement and loyalty across sites, and is built-in to Cisco Meraki APs with no additional cost or complexity. Data collected by APs is synced with the Cisco Meraki cloud with  the flexibility to feed data into custom applications. Integrate with back-end CRM, drive in-store customer engagement through push notifications, or alert staff to VIP customer visits.\n\n**This solution doesn't need real-time Cisco Meraki setup as it's already got sample data required for end-to-end testing.**\n\n**For proper real-time setup please see the following link to get the integration working:**\nhttps://meraki.cisco.com/technologies/location-analytics-api\n\n## **Note:**\n\u003e This solution is not production ready as it requires security hardening and relevant scaling based on specific customer environments. \n\u003e Also, this will give you end-to-end plumbing of required Azure components to get started with basic visualistions / Insights, which can be extended based on requirements.\n\n## Use Case\n\nThis solution is a quick demonstration of the art of possible for any smart buildings, retail stores, universities, hospitals equipped with WAPs (Wireless Access Points) using Azure PaaS offering. It enables data ingestion omitted through user mobile devices into Azure Cloud via Meraki Cloud API for analytical purposes i.e. \n\n- Foot-Fall Analysis\n- Staff Optimisation\n- Store Layout Optimisation \n- Product Recommendation\n- Many other advance use cases\n\nAlong the way you will also be exposed to a number of other Azure components, namely Event Hub, Stream Analytics, and Power BI. When everything is successfully deployed and running, the final result will be a PowerBI dashboard showing the following Line chart for real time footfall captured via user mobile devices and further Dashboard visualisations can be added after batch aggregation / analysis.\n\n![dashboard](https://user-images.githubusercontent.com/31105197/29453710-7f8b1642-8402-11e7-9f3f-99b90d96c39c.png)\n\n\u003e **Note:** This solution is pre-configured with the Cisco Meraki settings but can be very easily tailored for other manufacturers. \n\n## Requirements\n\n- Microsoft \u003ca href=\"https://azure.microsoft.com/en-us/\"\u003eAzure\u003c/a\u003e subscription with login credentials\n- \u003ca href=\"https://powerbi.microsoft.com/\"\u003ePowerBI\u003c/a\u003e subscription with login credentials\n- A local installation of \u003ca href=\"https://azure.microsoft.com/en-us/documentation/articles/sql-data-warehouse-install-visual-studio/\"\u003eVisual Studio 2015 with SQL Server Data Tools (SSDT)\u003c/a\u003e\n- Azure SDK\n- Nuget Package installtion of \"WindowsAzure.ServiceBus\" \u0026 \"WindowsAzure.Storage\" from within Visual Studio 2015 --\u003eProject--\u003eManage NuGet Packages menu option\n\n# High Level Architecture\n\n![merakidatavisiorevised](https://user-images.githubusercontent.com/31105197/29423338-3d5d95e0-8373-11e7-804c-316d3e005c94.jpg)\n\n## Deploy\n\nBelow are the steps to deploy the use case into your Azure subscription.\n\n**Clone or download the code from Github repositry and open the solution file in Visual Studio 2015. It will look like the following:**\n\n![solution-image](./media/solutionimage.png)\n\n\u003e **Note:** Deploy the Azure Components in the following order.\n\n## Azure Blob Storage\nhttps://docs.microsoft.com/en-us/azure/storage/storage-introduction#blob-storage\n\n1. Parameters\n   1. Right-Click on MerakiStorage Project in Solution Explorer \n   1. Change: LOCATION: **[*LOCATION*]** (The region where everything will be deployed) and Other properties as needed\n   1. Click: Edit Paramters (If you need to change default values)\n   1. Click: **OK**\n\n\n## Azure SQL (Batch Analytics / Processing)\nhttps://docs.microsoft.com/en-us/azure/storage/storage-introduction#blob-storage\n\n1. Parameters\n   1. Right-Click on MerakiSQL Project in Solution Explorer \n   1. Change: LOCATION: **[*LOCATION*]** (The region where everything will be deployed) and Other properties as needed\n   1. Click: Edit Paramters (If you need to change default values)\n   1. Click: **OK**\n\nNext you need to create the following table in the AZURE SQL DB. You can do this by following these steps:\n\n1. Start Visual Studio. Note that you must have installed the SQL Server Data Tools.\n1. Select: View: **SQL Server Object Explorer**\n1. Right click: **SQL Server**\n1. Click: **Add SQL Server...**\n1. Type: Server Name: **merakiservertest.database.windows.net**\n1. Select: Authentication: **Sql Server Authentication**\n1. Type: User name: **CiscoTest**\n1. Type: Password: **Meraki121121#**\n1. Select: Database Name: **merakidb**\n1. Click: **Connect**\n1. Right click: **merakidb**\n1. Select: **New Query...**\n1. Copy and paste:\n\n CREATE TABLE [dbo].[tblMeraki] (\n    [Id]           INT          IDENTITY (1, 1) NOT NULL,\n    [Version]      VARCHAR (5)  NULL,\n    [Secret]       VARCHAR (50) NULL,\n    [Type]         VARCHAR (20) NULL,\n    [apFloors]     VARCHAR (50) NULL,\n    [apTags]       VARCHAR (50) NULL,\n    [apMac]        VARCHAR (50) NULL,\n    [manufacturer] VARCHAR (50) NULL,\n    [lng]          DECIMAL (18) NULL,\n    [lat]          DECIMAL (18) NULL,\n    [x]            DECIMAL (18) NULL,\n    [y]            DECIMAL (18) NULL,\n    [unc]          DECIMAL (18) NULL,\n    [seentime]     DATETIME     NULL,\n    [ssid]         VARCHAR (50) NULL,\n    [os]           CHAR (10)    NULL,\n    [clientMac]    VARCHAR (50) NULL,\n    [seenEpoch]    VARCHAR (50) NULL,\n    [rssi]         VARCHAR (50) NULL,\n    PRIMARY KEY CLUSTERED ([Id] ASC)\n);\n\n1. Click: **Execute**\n\n## Azure Event Hubs \nhttps://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-what-is-event-hubs\n\nThis is currently created with 1 Throughput Unit.\n\n1. Parameters\n   1. Right-Click on MerakiEventHub Project in Solution Explorer \n   1. Change: LOCATION: **[*LOCATION*]** (The region where everything will be deployed) and Other properties as needed\n   1. Click: Edit Paramters (If you need to change default values)\n   1. Click: **OK**\n\n## Azure Web App\n\n**Before deploying the web app go to Event Hubs in the portal grab the Connection String Primary from under Consumer groups section by clicking on Shared access policies--\u003e ManagePolicy**\n\n![ManagePolicy-image](./media/ManagePolicy.png)\n\n**After grabbing that paste that in the \u003cMerakiCisco\u003e project within Handler.ashx file where the Event Hub connection string currently applied, finally, Rebuild the project for the settings to apply.** \n\nNow, follow the steps below to deploy the MerakiWebApp project:\n\n1. Parameters\n   1. Right-Click on MerakiWebAPP Project in Solution Explorer \n   1. Change: LOCATION: **[*LOCATION*]** (The region where everything will be deployed) and Other properties as needed\n   1. Click: Edit Paramters (If you need to change default values)\n   1. Click: **OK**\n\n## Azure Stream Analytics\nhttps://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-introduction\n\n1. Parameters\n   1. Right-Click on MerakiStreamAnalytics Project in Solution Explorer \n   1. Change: LOCATION: **[*LOCATION*]** (The region where everything will be deployed) and Other properties as needed\n   1. Click: Edit Paramters (If you need to change default values)\n   1. Click: **OK**\n\n**For further details on how to create Inputs/Outputs for Stream Analytics refer to following links**\nhttps://docs.microsoft.com/en-gb/azure/stream-analytics/stream-analytics-define-inputs#create-data-stream-input-from-event-hubs\nhttps://docs.microsoft.com/en-gb/azure/stream-analytics/stream-analytics-define-outputs\n\nBrowse: https://portal.azure.com\n1. Click: **STREAM ANALYTICS** \u003e **streamanalyticsjob[*unique*]** \u003e Click on Query \u003e Goto solution explorer look for StreamAnalyticsJobQuery.sql file under scripts folder \u003e Copy and paste that in the Query window\n1. Select: Add Inputs / Outputs based on following diagram with same **names specified** to match the query outputs\n1.  Click: **Finish** \u003e **Start** \u003e **Finish** (You do not need to specify a custom time)\n\n![StreamAnalytics-image](./media/streamanalytics.png)\n\n## Test\n\n**In Order to confirm that all the above plumbing of Azure components is working as expected before visualising the Real-time output in Power BI would required a quick test as following:**\n\nDownload or Install either Fiddler / Postman or any web debugger tool for creating Http GET/POST requests:\nhttps://www.telerik.com/download/fiddler\n\n- Goto Solution explorer and copy the test data from \"merakitestdata.json\" file under scripts and paste in the Request Body of the fiddler tool as shown in the following image:\n- Get the ** deployed MerakiCisco website URL** from Azure Portal for the deployed web app and paste it in the URL section under \"Composer Tab\" next to POST method (as highlighted in the image)\n![fiddler-image](./media/fiddler.png)\n\n- If the test shows \"Status\" 200 as shown in the image then all the integration worked as expected and outputs can be visualised using the following Power BI Setup.\n\n## Create the PBI dashboard\n\n### Realtime visualization\n\n1. Browse: https://powerbi.microsoft.com\n1. Click: **Sign in** (Login with your credentials)\n1. Show: The navigation pane\n1. Click: **meraki** (Under the Datasets folder \u003e **Line chart** # Under Visualizations)\n1. Drag: **seentimeime**: To: **Axis**\n1. Drag: **apfloors**: To: **Legend**\n1. Drag: **clientmac**: To: **Values**\n1. Click: **Save**\n1. Type: Name: **MerakiReport**\n1. Click: **Save** \u003e **Pin visual** (pin icon on upper-right)\n1. Select: **New dashboard**\n1. Type: Name: **MerakiDashboard**\n1. Click: **Pin**\n\n## TODOs\n- Use Https for communication between Meraki API and Web APP Endpoint (Currently going through a process to get it implemneted with the help of Cisco)\n- Streamline script outputs\n- Add Batch Processing Power BI Visualisations for interesting insights i.e. floor plans mapping etc.\n- Add error handling\n- Automate Stream Analytics Input/Output and Query sections\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaliraaza%2FMerakiAnalytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faaliraaza%2FMerakiAnalytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaliraaza%2FMerakiAnalytics/lists"}