{"id":29604794,"url":"https://github.com/kweinmeister/agentitest","last_synced_at":"2025-07-20T16:02:20.557Z","repository":{"id":304836435,"uuid":"1010200407","full_name":"kweinmeister/agentitest","owner":"kweinmeister","description":"Write browser tests in natural language, built on pytest and allure.","archived":false,"fork":false,"pushed_at":"2025-07-15T13:32:23.000Z","size":429,"stargazers_count":6,"open_issues_count":0,"forks_count":4,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-16T06:22:09.451Z","etag":null,"topics":["agent","ai","allure-report","browser-use","gemini","llm","playwright","pytest","qa","test-automation","testing","web-testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kweinmeister.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-28T15:09:19.000Z","updated_at":"2025-07-15T13:32:27.000Z","dependencies_parsed_at":"2025-07-16T09:50:12.332Z","dependency_job_id":"d7fcd8cf-b894-4028-a42a-6021852642ff","html_url":"https://github.com/kweinmeister/agentitest","commit_stats":null,"previous_names":["kweinmeister/agentitest"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/kweinmeister/agentitest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kweinmeister%2Fagentitest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kweinmeister%2Fagentitest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kweinmeister%2Fagentitest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kweinmeister%2Fagentitest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kweinmeister","download_url":"https://codeload.github.com/kweinmeister/agentitest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kweinmeister%2Fagentitest/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266152251,"owners_count":23884473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","ai","allure-report","browser-use","gemini","llm","playwright","pytest","qa","test-automation","testing","web-testing"],"created_at":"2025-07-20T16:00:42.123Z","updated_at":"2025-07-20T16:02:20.551Z","avatar_url":"https://github.com/kweinmeister.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤖 AgentiTest: Agentic Test Automation\n\nThis framework uses an LLM to automate browser testing through high-level, natural language commands. It's designed to be a flexible and developer-friendly tool for creating robust and maintainable browser tests.\n\n## 🛠️ Tech Stack\n\n* **[`browser-use`](https://github.com/browser-use/browser-use)**: Translates natural language into browser actions with an agent.\n* **[`pytest`](https://github.com/pytest-dev/pytest)**: A framework for structuring and running tests.\n* **[Allure Report](https://github.com/allure-framework/allure2)**: For creating interactive and detailed test reports.\n\n![Allure Report Overview](media/allure-report-overview.png)\n\n## ✨ Key Features\n\n* **Natural Language Tests**: Write tests using descriptive, natural language tasks instead of brittle selectors.\n* **Adaptable Logic**: The core testing logic is designed to be easily adapted for any website.\n* **Insightful Reports**: Allure reports provide screenshots, agent actions, and AI \"thoughts\" for each step, giving you a clear view into the test execution.\n* **Environment Documentation**: Key details about the test environment—such as the OS, Python version, and browser version—are automatically included in the Allure report for better debugging and context.\n* **Customizable Browser Settings**: The framework allows for detailed configuration of browser settings. For more information, see the [Browser Settings documentation](https://docs.browser-use.com/customize/browser-settings).\n* **Example Implementation**: `test_community_website.py` provides a practical example of how to write a test suite.\n\n## ⚙️ How It Works\n\nThe framework's structure is designed for clarity and scalability, separating shared logic from individual tests. It's composed of two main parts:\n\n* **A common file (`conftest.py`)**: This central file contains `pytest` fixtures for setting up the browser and the LLM, along with a reusable `run_agent_task` helper function that executes test steps.\n* **Individual test files (`test_*.py`)**: These files contain your actual tests. You import and use the `run_agent_task` function to define test scenarios in natural language, focusing on what to test, not how.\n\n![Detailed Test Execution](media/allure-report-details.png)\n\n## ✍️ Writing Your Own Tests\n\nCreating a new test is straightforward. All tests should inherit from the `BaseAgentTest` class, which provides the necessary setup for running agent-based tests.\n\nHere’s a minimal example of a test file:\n\n```python\nimport allure\nimport pytest\nfrom conftest import BaseAgentTest\n\n@allure.feature(\"Login Functionality\")\nclass TestLogin(BaseAgentTest):\n    \"\"\"Tests for the website's login functionality.\"\"\"\n\n    @allure.story(\"Successful Login\")\n    @allure.title(\"Test Successful Login\")\n    @pytest.mark.asyncio\n    async def test_successful_login(self, llm, browser_session):\n        \"\"\"Tests that a user can successfully log in.\"\"\"\n        task = \"click the login button, fill in the username and password, and confirm that the user is logged in.\"\n        await self.validate_task(llm, browser_session, task, \"Welcome, user!\")\n```\n\nIn this example:\n\n* `TestLogin` inherits from `BaseAgentTest`.\n* The `@pytest.mark.asyncio` decorator tells pytest that this is an asynchronous test, which is necessary because the browser automation library (`browser-use`) is asynchronous.\n* The test method `test_successful_login` is an `async` function that accepts `llm` and `browser_session` fixtures.\n* `self.validate_task` is called with the task description and the expected result.\n\nThis structure allows you to write tests that are easy to read and maintain, focusing on the user's actions and the expected outcomes.\n\nFor more information on how to use Allure with pytest, see the [official Allure documentation](https://allurereport.org/docs/pytest).\n\n## 🚀 Setup and Installation\n\n1. **Create a Virtual Environment**:\n\n    ```bash\n    python -m venv venv\n    source venv/bin/activate  # On Windows, use `venv\\Scripts\\activate`\n    ```\n\n2. **Install Dependencies**:\n\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n3. **Set Up Environment Variables**:\n\n    To get started, create a `.env` file in the root of the project by copying the provided template:\n\n    ```bash\n    cp .env.example .env\n    ```\n\n    Next, open the `.env` file and add your `GOOGLE_API_KEY`. This file is pre-configured with sensible defaults, but you can customize them to fit your needs.\n\n    **Required Variables**:\n    * `GOOGLE_API_KEY`: Your Google API key for accessing the Gemini model.\n\n    **Optional Variables**:\n    * `GEMINI_MODEL`: The specific Gemini model you want to use (e.g., `gemini-2.5-pro`). For a list of available models, see the [Gemini models documentation](https://ai.google.dev/gemini-api/docs/models).\n    * `HEADLESS`: Set to `true` to run in headless mode (without a visible browser UI) or `false` to run with a visible UI.\n\n## 🧪 Running the Tests\n\nOnce your environment is configured, you can run the tests and generate the Allure report data with a single command:\n\n```bash\npytest\n```\n\nThis will create an `allure-results` directory containing the data for your test report. The output directory is specified by the `--alluredir` parameter in `pytest.ini`, which is required for Allure to function correctly. For more details, see the [Allure pytest documentation on `alluredir`](https://allurereport.org/docs/pytest-configuration/#alluredir-%E2%9F%A8directory%E2%9F%A9).\n\n### Customizing Test Execution with `pytest.ini`\n\nThe `pytest.ini` file allows you to customize test execution. For example, you can add default command-line options or define custom markers. For more details, see the [official pytest documentation](https://docs.pytest.org/en/stable/reference/customize.html).\n\n## 📊 Viewing the Allure Report\n\nTo view the interactive Allure report, first make sure you have Allure installed (`npm install -g allure-commandline`), and then run:\n\n```bash\nallure serve allure-results\n```\n\nThis will start a local web server and open the report in your browser to view the test results.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkweinmeister%2Fagentitest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkweinmeister%2Fagentitest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkweinmeister%2Fagentitest/lists"}