Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/hjsblogger/canvas-automation-selenium-cypress

This repo contains Cypress and Selenium Python examples for automating interaction with Canvas Elements
https://github.com/hjsblogger/canvas-automation-selenium-cypress

automation canvas canvas-game cypress cypress-cloud selenium selenium-cloud selenium-java selenium-python selenium-webdriver web-automation

Last synced: about 1 month ago
JSON representation

This repo contains Cypress and Selenium Python examples for automating interaction with Canvas Elements

Awesome Lists containing this project

README

        

Banner

This repo contains implementation for automating interactions with Canvas elements using Selenium (+ Python) and Cypress. The tests are executed on the [LambdaTest cloud grid](http://www.lambdatest.com?fp_ref=himanshu15), but the same tests can also work with the respective test frameworks installed on the local machine.

Here are the overall usecases covered in the repo:

## Canvas Element Handling With Selenium Python

Follow the steps along to trigger the test execution:

**Step 1**

Create a virtual environment by triggering the *virtualenv venv* command on the terminal

```bash
virtualenv venv
```
VirtualEnvironment

**Step 2**

Navigate the newly created virtual environment by triggering the *source venv/bin/activate* command on the terminal

```bash
source venv/bin/activate
```

**Step 4**

Install the dependencies by triggering the ```pip3 install -r requirements.txt``` command on the terminal.

selenium-chart-execution-1

#### Cloud Execution

**Step 3**

Procure the LambdaTest User Name and Access Key by navigating to [LambdaTest Account Page](https://accounts.lambdatest.com/security). You might need to create an an account on LambdaTest since it is used for running tests on the cloud Grid.

LambdaTestAccount

Set LambdaTest **Username** and **Access Key** in environment variables.

* For **Linux/macOS**:

```bash
export LT_USERNAME="LT_USERNAME"
export LT_ACCESS_KEY="LT_ACCESS_KEY"
```
* For **Windows**:
```bash
set LT_USERNAME="LT_USERNAME"
set LT_ACCESS_KEY="LT_ACCESS_KEY"
```

For demonstration, I am extracting tool-tip text from a bar-graph located in [CanvasJS](https://canvasjs.com/) website.

CanvasBarGraph


Here, I have used Selenium and the PyUnit (or unittest) framework in Python.

The example code is available in the [following](https://github.com/hjsblogger/canvas-automation-selenium-cypress/tree/main/selenium-python-demo) location. Run the command ```python3 canvas_bar_graph.py``` command on the terminal to extract content from the bar-graph.

selenium-bar-chart-1

selenium-bar-chart-2

Once the execution is triggered, navigate to the [LambdaTest automation dashboard](https://automation.lambdatest.com/) to check the status of the tests.

selenium-bar-graph-lambdatest-1

selenium-bar-graph-lambdatest-2

As seen above, the tool-tip text from the Canvas element (i.e. bar graph) was successfully read using Selenium Python.

#### Local Execution

Automating [LambdaTest Bug Smasher](https://www.lambdatest.com/bug-smasher) game play with [PyAutoGUI](https://github.com/asweigart/pyautogui). PyAutoGUI is a cross-platform GUI automation Python module used for programmatically controlling the mouse & keyboard. Please note that you might need to adjust the co-ordinate positions in case the clicks are not occurring at the right positions!

Trigger the command ```python3 canvas_autogui.py``` on the terminal to automate game-play using [PyAutoGUI](https://github.com/asweigart/pyautogui).

pyautogui-1

Here are some game-play screenshots that I captured using the Phone camera, since the browser window was in control of PyAutoGUI!

![pyauto-gui-2](https://github.com/user-attachments/assets/9dff1afd-e242-4898-8176-0966ffcdf440)

![pyauto-gui-3](https://github.com/user-attachments/assets/34738d93-761c-4bb4-b25d-07b930b04d9b)

With this, the game play automation (local Cypress) and bar-graph data extraction (Cypress cloud) is successful.

## Canvas Element Handling With Cypress

Automating [LambdaTest Bug Smasher](https://www.lambdatest.com/bug-smasher) game play with Cypress on the LambdaTest cloud grid. The automated game play also works seamlessly on Cypress installed on local machine (i.e. macOS Sequoia 15.1). The resolution & viewport size is set to 1024*768.

Follow the steps along to trigger the test execution:

**Step 1**

Create a virtual environment by triggering the *virtualenv venv* command on the terminal

```bash
virtualenv venv
```
VirtualEnvironment

**Step 2**

Navigate the newly created virtual environment by triggering the *source venv/bin/activate* command on the terminal

```bash
source venv/bin/activate
```

#### Cloud Execution

**Step 3**

Procure the LambdaTest User Name and Access Key by navigating to [LambdaTest Account Page](https://accounts.lambdatest.com/security). You might need to create an an account on LambdaTest since it is used for running tests on the cloud Grid.

LambdaTestAccount

Set LambdaTest **Username** and **Access Key** in environment variables.

* For **Linux/macOS**:

```bash
export LT_USERNAME="LT_USERNAME"
export LT_ACCESS_KEY="LT_ACCESS_KEY"
```
* For **Windows**:
```bash
set LT_USERNAME="LT_USERNAME"
set LT_ACCESS_KEY="LT_ACCESS_KEY"
```

**Step 4**

For execution on LambdaTest, add the credentials (i.e. LambdaTest username & access-key) to the [following lines](https://github.com/hjsblogger/canvas-automation-selenium-cypress/blob/main/cypress-demo/lambdatest-config.json#L3).

**Step 5**

Now that the credentials are all set, trigger the command ```lambdatest-cypress run``` on the terminal to run Canvas element tests on LambdaTest.

cypress-lambdatest

Once the execution is triggered, navigate to the [LambdaTest automation dashboard](https://automation.lambdatest.com/) to check the status of the tests.

cypress-lambdatest-execution

Shown below is the snapshot of the test execution where the game play is automated using Cypress (on LambdaTest cloud grid):

game-play-cypress-1

game-play-cypress-2

As seen above, I scored 85 points in the game :) Not a bad attempt at all :D.

#### Local Execution

**Step 3**

You can run the same test on Cypress (local machine) with Chrome (headed mode) browser by triggering the command ```npx cypress run --browser chrome --headed``` on the terminal. Please note that the Canvas implementation might require changes if the tests have to be executed on a different browser (e.g. Firefox, etc.).

**Step 4**

Shown below are some screenshots from game play automated on local machine:

cypress-local-execution

game-play-canvas-local-1

game-play-canvas-local-2

game-play-canvas-local-3

## Need Assistance?

Feel free to fork the repo and contribute to make it better! Email to [himanshu[dot]sheth[at]gmail[dot]com](mailto:[email protected]) for any queries or ping me on the following social media sites:

Twitter: [@hjsblogger](https://www.twitter.com/hjsblogger)


LinkedIn: [@hjsblogger](https://linkedin.com/in/hjsblogger)