# How to integrate with Asana Source: https://docs.qawolf.com/qawolf/Asana-2a15b2a994fb805287acdba6373e6758 Connect QA Wolf to Asana to automatically create and sync bug report issues in your Asana workspace. Select **Workspace settings** from the `Workspace name` dropdown above the flows list. Click **Integrations** from the left-side navigation bar. Click **Enable** next to Asana. Click **Connect** on the modal. Enable **Bug reports** and select the **Workspace** and **Project** in which bug reports are going to be created. * Click **Disconnect** to disable the integration. # Create flows with AI Source: https://docs.qawolf.com/qawolf/Automate-flows-using-AI-2d35b2a994fb8084b524cfc9aca7c358 Use AI flow creation to create, edit, and debug flows from plain-language or voice instructions. AI flow creation lets you create, edit, and debug flows by describing what you want in plain language or by speaking your instructions. You don't need to know the flow syntax to get started — the AI writes and updates the code for you. If you'd prefer to write flows manually, see [Create flows in the IDE](/qawolf/Create-a-new-flow-2c15b2a994fb802193a5c05e0219eb59). ## Start a flow Two entry points on the **Flows** tab hand a new flow to the agent: * **Create flow** — the button above the flows list * **Create flow** — the large purple button on the **All flows** page Both open the AI chat panel in flow mode. Working with the agent one prompt at a time is [guided creation](/qawolf/getting-the-most-out-of-automate#guided-creation). The icon next to **Create flow** opens a drawer with **Outline flows**, which starts an outlining session across a whole application at once — see [Create a coverage outline](/qawolf/Map-your-app-s-workflows-2a45b2a994fb80ca8a87c8fdc2a443fc). The Outline flows drawer next to the Create flow button ## Describe the flow you want Either **Create flow** button opens the AI chat panel in flow mode, offering **Create a flow** and **What should I test?** to start from. The AI chat panel in flow mode ### Type a description Type a plain-language description of what you want the flow to do into the chat box and hit enter. For example: *"Log in with a valid email and password and assert that the dashboard loads."* ### Speak a description Click the icon and speak your instructions. The agent will create the required groups and flows if they don't already exist. ## What happens next The agent generates the flow code and opens it in the editor. From here you can: * **Run it immediately** using the **Run** button to see it execute in the browser panel * **Review the code** if you want to understand or manually adjust what was generated * **Ask the AI to change it** by typing a follow-up instruction in the chat panel ## Complete flows automatically Select **Autonomously complete this test** to have the AI generate Playwright code for the flow automatically. While it runs, you can multitask in other flow tabs. ## Debug a failing flow When a flow fails, click **Fix with AI** in the banner at the top of the editor. Describe the problem or let the AI interpret the error automatically. For manual debugging, see [Debug tests & flows](/qawolf/Debug-tests-and-flows-2c15b2a994fb805c9e92eec7a4a3e83e). ## Import a third-party library In the AI chat panel, ask the agent to import the package you need. It will handle the entire process automatically. # How to integrate with Azure DevOps Boards Source: https://docs.qawolf.com/qawolf/Azure-DevOps-Boards-29c5b2a994fb81b09697f2f1e4f9fdf1 Connect QA Wolf to Azure DevOps Boards to automatically create and sync bug report issues using OAuth. Go to `https://dev.azure.com/ORGANIZATION_NAME/_settings/organizationPolicy` — **ORGANIZATION\_NAME** is your Azure DevOps Organization name. **Because** QA Wolf connects to Azure DevOps using OAuth, you must enable **Third-party application access via OAuth**. If **Third-party application access via OAuth** is disabled, an Azure DevOps organization owner must enable it before QA Wolf can be authorized. If you are not an organization owner, contact your Azure DevOps administrator. Select **Workspace settings** from the **Workspace name** dropdown above the flows list. Click **Integrations** from the left-side navigation bar. Click **Enable** next to Azure DevOps. Click on the connect button. Select the **Azure DevOps** service. Enter the Azure DevOps **Organization name** and click **Continue**. Click **Configuration**, then select the target project from the dropdown. * Click **Disconnect** at any time to disable the integration. # Overview Source: https://docs.qawolf.com/qawolf/CI-CD-Overview QA Wolf integrates with your CI pipeline to run tests automatically. Choose the outcome that matches your use case. Run tests when a new build is deployed to an environment. Run tests against a new mobile build artifact from your CI pipeline. # Choose the right CI environment to execute your tests Source: https://docs.qawolf.com/qawolf/Choose-the-right-CI-environment-to-execute-your-tests-29c5b2a994fb8031a3ddc1355eb4e7be QA Wolf supports testing across any CI environment, so you can align testing goals with your existing release process. Match your testing goals to the environment you run your tests in. ### Staging: When test stability matters most Use your **Staging** (QA or Test) environment for the best balance between reliability and safety. **What this gives you** * Stable builds with predictable data. * Space for complete end-to-end and regression testing. * Safe configuration for allowlisted email domains or test credentials. Staging may lag behind active development. If releases are infrequent, regressions might not surface until later in testing. ### Development: Get rapid feedback during test development Run tests in your **Development** environment to identify regressions early. **What this gives you** * Immediate feedback after merges. * Quick iteration and faster bug turnaround. * Tight integration with the development process. Development environments can be unstable or inconsistent. Failing tests here often point to temporary issues rather than actual defects. ### Production: Verify the real user experience Run tests in **Production** when you need complete confidence in real-world behavior. **What this gives you** * Testing against the exact production configuration. * Real performance and reliability data. * Confirmation that core user flows work correctly in the live environment. Bugs discovered here have already reached users. Some flows (such as payments) may trigger real transactions or pose a financial risk. ### Preview environments: Validate changes before merging code Ask QA Wolf to help you enable **PR testing** to run your flows automatically on each pull request in ephemeral preview environments. **What this gives you** * Early, automated feedback in your VCS. * Confidence that merges won't break mainline tests. * Faster, safer release cycles. PR testing requires dedicated, isolated environments. Shared resources or reused credentials can cause flaky results. QA Wolf recommends using [release candidates (RCs)](https://www.qawolf.com/blog/rcs-over-prs-how-rc-testing-aligns-velocity-and-quality) for PR testing. # Clear a release Source: https://docs.qawolf.com/qawolf/Clear-a-release-2c55b2a994fb808fa5ddfa157a89adcd Resolving failures is about deliberately clearing a release, not about achieving 100% passing results. An environment enters **Needs investigation** when one or more flows fail in a run. An environment enters **Failed** status when one or more flows mapped to high-priority bug reports fail. These states indicate that one or more runs require resolution before the release should proceed. Failed flows and environment status provide a release signal. If you want runs in *Failed* status to block releases, ask your Customer Success Manager about our CI Greenlight feature. A run that contains failed flows remains in **Investigating** or **Failed** as long as any failed flow has not been resolved. A failed flow is considered resolved when one of the following occurs: * The flow passes on a subsequent attempt, including when a temporary environmental issue resolves. * The flow is fixed, and the updated version is applied to the run. * The underlying application issue causing the failure is fixed. * A maintenance report is filed, preventing the flow from being included in runs until it is fixed. A run transitions to *Completed* only after all failed flows have been resolved. Once all runs are completed, the environment exits the *Needs investigation* or *Failed* status and can be considered ready for release. ## Reattempt a failure caused by a flake Reattempting is appropriate when all failures in the run are believed to be caused by a flake or a temporary environment glitch. Open the **Runs** tab and click the run that is in **Investigating** or **Failed** status to open the Investigation view. Click the icon to open the context menu. Runs Investigating Choose one of the following options depending on where you are in your investigation: * **Reattempt with latest version** — runs only the failed flows. Use this when you want to quickly check that a suspected temporary glitch has resolved. * **Duplicate run** — a full retry of all flows, including up to three attempts. Use this after you have validated that the temporary problem has resolved but want to check whether it was masking deeper issues. ## Bypass a failing flow to clear a release Bypassing a failed flow is appropriate when the failure is understood and should not block the release — for example, an understood failure in a third-party dependency. Bypassing does not fix the flow or the underlying issue. Instead, it records that the failure has been reviewed and does not require further investigation for the current release. Open the **Runs** tab and select a run in **Investigating** or **Failed** status. Select the failed run that contains the flow that doesn't need further investigation. Click **Diagnose** in the center panel. Select **Do not investigate**. Once a failed flow is marked as **Do not investigate**, it is considered resolved for the run. When all failed flows in the run have either passed or been bypassed, the run transitions to **Completed**. You can also find **Diagnose** by hovering over the failing flow to show the icon, then clicking it to show the icon. **Diagnose** will appear below the flow list. ## Repair a broken test to unblock a release From the **Runs** tab, select the environment from the list on the upper left. In the center panel, select a run from the **Investigating** list. In the Investigation view, click **Edit code** to open the flow for changes. Click **Run code**. When the flow fails, click **Fix with AI** in the banner at the top of the editor. Describe the problem or let the AI interpret the error automatically. You can also repair the test manually by modifying the code yourself. When satisfied with the changes, review and publish your changes. You **must** publish your changes to enable **Use as reattempt**. Click **Use as reattempt**. The flow should pass and will move to the **Completed** section of the Runs tab. ## Clear the release when there's a bug or broken test that can't be fixed immediately Some failures represent real issues but are not practical to fix during the current release. In these cases, the appropriate action is to document the work required and allow the release to proceed. Follow-up work generally falls into two categories: **For bugs** — [file a bug report](/qawolf/Manage-bug-reports-2c75b2a994fb80308ddcc7cc60703c2f). When you file a bug report, the severity is used to display warnings about the safety of a given release. **For flow maintenance** — [file a maintenance report](/qawolf/Manage-maintenance-reports-2d55b2a994fb80728909f48b1023f7fa). This prevents the flow from being executed in runs until it is fixed, which resolves the investigation state without requiring an immediate fix. # Create flows in the IDE Source: https://docs.qawolf.com/qawolf/Create-a-new-flow-2c15b2a994fb802193a5c05e0219eb59 Create flow files and folders and write test code manually in the QA Wolf IDE. If you'd prefer to have AI create and edit flows for you, see [Create flows with AI](/qawolf/Automate-flows-using-AI-2d35b2a994fb8084b524cfc9aca7c358). ## Create a folder or flow Two entry points on the **Flows** tab create a flow in the IDE. **From the flows list** Click in the toolbar, then select **New flow** or **New folder**. The New flow and New folder menu in the flows list toolbar **From the IDE** From the **Flows** tab, hover over the left sidebar to expand it, then click **IDE**. The left sidebar with the IDE view selected Navigate to the location in the file tree where you want to create the new file or folder. Click the icon in the toolbar. The IDE file explorer Click **New Folder** or **New Flow**. Use clear, descriptive names that reflect your app's feature areas — this helps both you and AI keep flows organized. ## Write test code Select a flow file from the tree to open it in the code editor. The editor prefills new flows with [AAA](https://www.qawolf.com/blog/intro-to-aaa) (Arrange-Act-Assert) comments as a starting structure. For a full explanation of flow structure and imports, see [Anatomy of a flow](/qawolf/anatomy-of-a-qa-wolf-test-mobile-edition). QA Wolf lints your code as you type. See [Debug tests & flows](/qawolf/Debug-tests-and-flows-2c15b2a994fb805c9e92eec7a4a3e83e) if you encounter errors. ## Publish your changes When ready, click **Review**. Unpublished Review your changes in the side-by-side diff. Publish Enter a clear description of the change in the **Adding a comment** field. Click **Publish changes** to confirm. The publish dialog will close when your changes have been saved. # How to verify and debug your test Source: https://docs.qawolf.com/qawolf/Debug-tests-and-flows-2c15b2a994fb805c9e92eec7a4a3e83e Run your test code in a live browser, identify failures, and use debugging tools to fix issues. ## Run your flow Click **Run** to execute the flow. A live browser appears in the right panel where you can watch the code execute in real time. A flow running in the IDE When the flow completes successfully, a green **Flow passed** banner appears at the top of the editor. ## Debug a failure If something goes wrong, a red **Flow failed** banner appears at the top of the editor and the failed line is highlighted in red. A failed flow in the IDE You have two options for fixing it: **Fix with AI** — click **Fix with AI** in the banner to let the agent diagnose and repair the failure automatically. You can also describe the problem in the AI chat panel or paste the error directly from the **Logs** tab. **Run selected lines** — reset the app state in the browser, select the lines you want to test, and click **Run**. This lets you iterate on a specific section without re-running the entire flow. ## Use the debugging panels The bottom-right panel has four tabs that provide different views of your flow's execution. **Problems** — the default tab when you open a flow. Appears when QA Wolf detects issues such as naming conflicts. Affected files and folders are marked with a yellow dot in the file tree. Resolve any problems listed before running or publishing to avoid confusing output. The Problems tab **Logs** — shows a timestamped execution log. When a flow fails, the error message here is the fastest way to understand what went wrong. You can paste the log output directly into the AI chat panel to get a fix. The Logs tab **Elements** — lets you enter a locator or CSS selector to highlight matching elements in the live browser. Useful for verifying selectors while writing or debugging a flow. See [Element picker](/qawolf/Use-the-Elements-picker-2c35b2a994fb80a681baca4f6acf3179) for more details. The Elements tab **Details** — shows tags and readiness status for the current flow. Both fields can be edited directly from this tab. After a run, switches to show individual run steps with timing, useful for identifying slow or hung steps. The Details tab ## Check run history The **Run History** tab shows a pass/fail chart and the result of the most recent run. The Run History tab Run History only shows results from scheduled runs. Manual runs from the editor or Flows tab do not appear here. See [How Runs work](/qawolf/Run-a-flow-or-suite-2a25b2a994fb8014b164d7ad2efd5ddel) for a full explanation of run types. ## Manage tags and readiness status The **Details** tab shows the tags and readiness status assigned to the current flow. Both fields can be edited directly from this tab using the icon next to each field. After a run, the tab switches to display individual run steps with timing, which is useful for identifying slow or hung steps. Run steps with timing in the Details tab # Diagnose the cause of a failing flow Source: https://docs.qawolf.com/qawolf/Diagnose-the-cause-of-a-failing-flow-2c65b2a994fb8034ae2bd0def1f8dfe5 Learn how to classify flow failures as flakes, bugs, or broken tests using the Investigation view. ## The different reasons flows fail When a flow fails, the next step is to determine why. Identifying the cause determines the appropriate resolution path and helps avoid unnecessary fixes or retries. Failures generally fall into one of the following categories: * **Flakes** — failures caused by infrastructure outages, network instability, or failed dependent services. These issues often resolve without changes to the flow or the application. * **Bugs** — failures that reveal incorrect or unexpected behavior in the application under test. These indicate a product issue rather than a problem with the flow itself. * **Broken tests** — failures caused by flow logic, bad selectors, incorrect assertions, or assumptions that no longer match the application. These may be simple to correct or may require more substantial maintenance. Correctly identifying the cause helps you determine whether the failing flow should be retried, fixed, deferred, or accepted to clear the release. ## Use the Investigation view to diagnose a failure The **Investigation view** is where you diagnose flows that failed during a scheduled or event-based run. It brings together everything you need to understand what happened in a single run and decide what to do next. You reach the Investigation view by opening a run from the **Runs** tab when that run is in **Investigating** or **Failed** status. The view opens with the most relevant information already selected, so you can start reviewing immediately. ### Find failing flows Navigate to the Investigation view. The left panel separates **Failed**, **Running**, and **Passed** flows. Click the **Failed** tab to see just those flows. ### Identify the failing line of code Navigate to the failing flows in the Investigation view. Select a flow. The point of failure is highlighted. ### Compare attempts in a single run If a flow was retried multiple times during a run, you can switch between attempts to determine whether the failure was consistent or occurred only under certain conditions. ### Review what happened in the browser during the most recent run Recorded video shows exactly what happened in the browser during the selected attempt, which is often the fastest way to understand unexpected behavior. ### Compare previous runs To review video of up to the last 100 runs of a flow, including all attempts, use the flow's **Run History** tab. ### Access Playwright traces Navigate to the failing flow from the Investigation view. Click the **Trace** tab on the right panel. ## Use Run History, Activity, and Notes to diagnose failures Some issues only become clear when you look beyond a single run. **Run History** shows the last 100 runs for a flow, including each attempt. Reviewing this history helps you spot patterns such as intermittent failures, regressions after changes, or flows that consistently fail in the same place. You'll typically access these tools from the Investigation view or the **Run History** tab in the IDE. Run History is also available in the IDE — select a flow and click the **Run History** tab in the right panel. ### Focus on a specific run Hover over the vertical bars in the **Run History** timeline. As you move across the bars, the selected bar expands to show the icon, and the timestamp below the timeline updates to match that run. Click the icon to open details for the selected run. ### Review change history The **Activity** tab shows changes made to a flow over time, including readiness updates and maintenance reports. This helps explain when a flow was modified or temporarily removed from runs and why. ### Record and review notes The **Notes** tab lets you record context about a flow, such as known issues, edge cases, or investigation findings. Notes are saved per flow and can help guide future diagnosis and maintenance. # Configure environments Source: https://docs.qawolf.com/qawolf/Environment-settings-2cd5b2a994fb8014ba8bd7aa25f3ed0f Create and manage environments in QA Wolf with separate variables, concurrency settings, and VPN access. ## Environments In the QA Wolf platform, an [environment](/qawolf/Glossary-29c5b2a994fb806d93f2ce4554a2fcfc) corresponds to a specific version of your app, such as development, staging, or production. Each environment can be configured with separate variables (e.g., base URL, users), concurrency rates, and other settings relevant to the application version you're testing. Flows and tests are unique to each environment and are not shared across environments. ## Create an environment Choose **Workspace settings** from the `Workspace name` dropdown above the flows list. Click **Environments** in the left navigation bar. Click **Create environment** in the upper-right corner. Give it a unique name and click **Create**. Replace the existing Environment Name and press Enter. ## Change your default environment Both the **Flows** and **Runs** tabs require a default environment. Set this to the environment you use most often for running and debugging tests. Open the `Workspace name` dropdown and click **Workspace settings**. Go to **General**. Under **Default environments**, choose the environment you want the **Flows** and **Runs** tabs to use by default from the **Default team environment** dropdown. If your team is using PR testing environments, select the default environment from the dropdown labeled **Default team PR Testing base environment**. Contact your QA Wolf Customer Success Manager to assist with PR testing configuration. ## Limit the environment's concurrency By default, QA Wolf executes all flows in a run concurrently. You can change this by capping concurrency per environment. Open the `Workspace name` dropdown and click **Workspace settings**. Click **Environments** in the left navigation bar. Scroll to **Run Concurrency Limit** and select the maximum number of flows to be executed at the same time. Click **Save**. ## Create and manage environment variables In the QA Wolf platform, environment variables are encrypted constants scoped to a specific environment. You can use the same variable across environments by reusing the same name in each environment and setting the appropriate value for that environment (for example, `BASE_URL` in Staging vs Production). QA Wolf doesn't enforce cross-environment consistency—this pattern relies on naming conventions. Open the `Workspace name` dropdown and click **Workspace settings**. Click **Environments** in the left navigation bar. Click the **Environment variables** tab. These can also be updated from the **Flows** tab. # Integrate a mobile build with Fastlane Source: https://docs.qawolf.com/qawolf/Fastlane-2db5b2a994fb802186a3d88573899c78 Upload Android and iOS build artifacts and trigger QA Wolf test runs from your existing Fastlane lanes using the QA Wolf plugin. ## When to use Fastlane This guide is for teams that already use Fastlane to build Android or iOS apps. If your mobile builds are defined as Fastlane lanes, the QA Wolf Fastlane plugin lets you upload build artifacts and trigger test runs within your existing mobile automation. This is the recommended option for mobile teams that use Fastlane today. If you do not use Fastlane, use the QA Wolf CI SDK instead. QA Wolf does not set up Fastlane or create build lanes—this guide assumes Fastlane is already installed and that it is producing a mobile build artifact. ## Before you begin Before configuring the Fastlane integration, make sure you have the following in place: * A working Fastlane setup for your Android or iOS app. * A Fastlane lane that produces a mobile build artifact (APK, AAB, or IPA). * A QA Wolf API key stored as a secret in your CI environment. * Artifact naming conventions are defined for your environments. See [Artifact naming conventions](#artifact-naming-conventions) below. Before mobile test runs can execute, QA Wolf must enable mobile triggers for your workspace. QA Wolf will handle this and may ask you for: * Which environments you want to test. * Whether PR testing is enabled. * The artifact naming conventions you are using. * The upload and trigger method you chose. Until this step is complete, CI jobs can upload artifacts and send deployment notifications, but mobile test runs will not start automatically. ## How Fastlane works with QA Wolf The Fastlane integration uses a QA Wolf Fastlane plugin to perform two actions: Upload a mobile build artifact to QA Wolf. Notify QA Wolf of a deployment event so a test run can be triggered. You can upload builds without triggering runs, which is useful during initial setup or validation. The artifact name you provide is used to associate the build with the correct environment. QA Wolf automatically applies the file extension. ## Install & configure Fastlane for QA Wolf Add the [QA Wolf Fastlane plugin](https://github.com/qawolf/fastlane-plugin-qawolf) to your project if it is not already installed. This is typically done in your **Fastfile** or plugin configuration, depending on how your project manages Fastlane plugins. Follow the instructions on the `README` to install. Make sure your CI environment exports the **QAWOLF\_API\_KEY** environment variable so Fastlane can authenticate with QA Wolf. ### Find the QAWOLF\_API\_KEY Open the `Workspace name` dropdown in QA Wolf and click **Workspace Settings**. Choose **Integrations**. Generate your **QAWOLF\_API\_KEY** by clicking the icon to the right of **API Key** under **API Access**. ### Add the QAWOLF\_API\_KEY secret Store `QAWOLF_API_KEY` as a secret in your CI system and ensure it is exported as an environment variable in the job that runs Fastlane. The exact steps depend on your CI provider — refer to your CI system's documentation for storing secrets. ## Artifact naming conventions Mobile build artifacts must follow consistent naming conventions so QA Wolf can correctly associate each build with the right environment and make failures easier to diagnose. The artifact name is used to identify: * Which environment the build belongs to * Whether the build is tied to a pull request * Which build was used for a given test run ### Static environments Static environments are long-lived environments such as staging or release environments. **Format** ```text theme={null} - ``` **Example** ```text theme={null} app-staging ``` Use the same basename every time a build is generated for the same environment. ### PR (ephemeral) environments PR environments are short-lived and tied to a specific pull request. These are only relevant if PR testing is enabled. **Format** ```text theme={null} ---pr ``` **Example** ```text theme={null} app-myorg-myrepo-pr123 ``` Including the organization, repository, and pull request number ensures each build can be traced back to the correct change and environment. QA Wolf applies the file extension (.apk, .aab, or .ipa) automatically based on the uploaded artifact. You only need to provide the basename. ## Upload a mobile build artifact Once your Fastlane lane produces a build artifact, you can upload it to QA Wolf. Add the `upload_to_qawolf` action to your lane and provide the artifact basename that matches your naming conventions. If this step completes successfully, the artifact has been uploaded and is available for use in test runs. ```ruby theme={null} lane :qawolf do # Build your app gradle( task: "assemble", build_type: "Release" ) # Upload the artifact to QA Wolf upload_to_qawolf( executable_file_basename: "app-staging" ) end ``` ## Trigger a test run After uploading a build artifact, you can notify QA Wolf that a new deployment is ready for testing. Add the `notify_deploy_qawolf` action to your lane. This step is optional and is required only if you want a test run to start automatically. The environment key must match the value configured by QA Wolf for your workspace. If mobile triggers have not yet been enabled, this step will complete without starting a test run. ```ruby theme={null} notify_deploy_qawolf( executable_environment_key: "RUN_INPUT_PATH" ) ``` ## Verify the integration Run the Fastlane lane in CI. Confirm the artifact upload step completes successfully. Confirm the deploy notification step runs without errors. Once mobile triggers are enabled, check the **Runs** tab to see the triggered test run. If no run appears, the most common cause is a missing or incomplete QA Wolf platform configuration. ## Troubleshooting and common issues * **Artifact uploads succeed, but no runs start:** Mobile triggers may not be enabled yet. Contact QA Wolf to complete platform configuration. * **Artifact not found during test execution:** Verify the artifact basename matches the expected naming conventions. * **Authentication errors:** Confirm QAWOLF\_API\_KEY is set correctly in your CI environment. * **Incorrect environment used:** Verify the environment key passed to notify\_deploy\_qawolf matches the configured value. # Full Service FAQ Source: https://docs.qawolf.com/qawolf/Full-Service-FAQs-29c5b2a994fb8000b442fa8aed05c9e5 Answers to common questions about QA Wolf full service, including test ownership, coverage, and how the service works. ## General ### What is QA Wolf's full service? QA Wolf's full service means we don't just provide the platform — we also create, maintain, and run your end-to-end tests for you. Our team writes Playwright and Appium tests, expands coverage over time, fixes broken tests as your product changes, and ensures tests run reliably on every pull request, schedule, or trigger you choose. You still own the code and can modify it at any time. We simply handle the ongoing work required to keep end-to-end coverage healthy. With full service, QA Wolf: * Writes and expands your end-to-end tests * Maintains tests as your app evolves * Runs tests in parallel across our infrastructure * Investigates failures and fixes test issues * Provides clear artifacts like videos, logs, and traces for every run The goal is simple: you get high end-to-end coverage without needing to build and operate a large internal test automation team. ### Who will we be working with from QA Wolf? Every customer gets a dedicated team of Wolves to make sure that they're successful: * **Customer Success Manager.** Strategic owner on QA Wolf's side for our partnership, helping ensure we are aligned on your goals and helping you get the most out of our service. * **QA Lead.** Primary point of contact for technical implementation, including your test plan, implementation, and testing of new features. * **QA Team.** Responsible for implementing tests, investigating failures, maintaining tests, and reporting bugs. * **Account Executive.** Your AE will guide you through contracting, security reviews, and any other internal procedures your company requires. ## Implementation ### In what order does QA Wolf build tests? Unless there are specific high-priority tests that need to be done first, we generally start with the most complex flows. These take longer to build and often require more collaboration with your team, so tackling them early helps speed up the rest of the process. Test creation isn't linear — we don't build one test after another in sequence. Because we focus on the hardest flows first, your suite will come together unevenly at first and then accelerate as simpler tests are added. If you have critical flows you'd like prioritized earlier, let your QA Lead know. They can adjust the build plan to align with your release schedule. ### How can I see how many flows your team has built? Open the Flows tab. Select **All Flows** from the left sidebar. The total number of groups and flows appears on the right panel at the top. #### How can I see the number of active flows your team has built? Set your filter using the icon. Hover over your team name in **All Flows** above to see the icon, then click it to get the icon. Below the list to the right, the number to the right of the icon shows the total number of flows, and the number to the right of the icon shows the total number of tests that meet the filter criteria. ### When will our tests start running? Tests start running as soon as they're built, and our goal is to deliver value fast — beginning with your most business-critical flows. To keep things moving: * Make sure your team is available for the **Product Tour** meeting. * Approve the **coverage outline** as soon as it's ready. Engage with us early, so we prioritize the right flows and start catching issues within the first few days of onboarding. ### What happens when a test fails? When tests fail, a QA Wolf engineer investigates the issue and determines whether it is a bug in the application or the test. If there's a bug in the application, we will file a bug report through your messaging system (e.g., Slack, Teams) and issue tracker (e.g., Jira, Linear). If there's a problem with the test, it can usually be resolved on the fly. However, some tests will need to be quarantined for more substantial maintenance work. You can monitor the [status of failed tests](/qawolf/Interpret-run-results-2d55b2a994fb80199754e4b2d3ce0318) in the Runs tab. ### How can we prevent planned application changes from blocking a release? One of the most common release blockers is a flow that requires maintenance before it can proceed. Most maintenance is simple and happens immediately, but sometimes a flow requires a more comprehensive update. When that happens, the affected flows are disabled and placed in **maintenance mode** until they can be fixed. If you're planning changes that will affect more than a few pages or significantly alter the DOM, let your QA Lead know ahead of time. They can usually update or refactor the flows in advance so runs stay smooth. When in doubt, just shoot us a quick message — even a heads-up helps prevent coverage gaps or blocked runs. ### How does QA Wolf handle flaky tests? As a customer, you'll never have to deal with flaky tests, which are tests that fail one or more times before eventually passing. Since flaky tests create noise that would otherwise slow your release velocity, our dedicated QA team will handle everything and ensure you see only real, human-verified bugs. That's our Zero Flake Guarantee. To keep things moving as quickly as possible, any test that fails is automatically re-run. That's because anything from a simple network hiccup to a slow email server could cause a flake. After multiple consecutive failures, a human steps in to investigate the root cause. ### How often will our test suite run? By default, your suite will run once a day, which is where most companies start. As their deployment processes mature, they increase testing frequency. We also support PR testing for [GitHub](/qawolf/PR-testing-for-GitHub-Integrations-2db5b2a994fb80a0aa94f3297d1f60c3) and [GitLab](/qawolf/PR-testing-for-GitLab-Integrations-2db5b2a994fb8075b13dd9d5c2e6be8c) through our [Scheduled Runs](/qawolf/Scheduling-flows-29c5b2a994fb80789f3fea32feae8f92). You can ask your QA Lead to increase or decrease your test runs to best fit your process. Unlimited test runs are included in your contract, so we encourage running your suite as frequently as it makes sense for your team and process. # Onboarding to QA Wolf Full Service Source: https://docs.qawolf.com/qawolf/Full-Service-The-First-Three-Months-29c5b2a994fb81bb9e10e4182639f294 What to expect during QA Wolf full service onboarding, including timelines, meetings, and setup steps. QA Wolf Full Service pairs your team with a dedicated QA engineering team that builds, runs, and maintains your automated test suite. If you're evaluating QA Wolf, [book a demo](https://www.qawolf.com/book-a-demo) to talk through your coverage goals. ## Onboarding Onboarding with QA Wolf is easy, but a few things need to be set up before we can begin developing a Test Plan and building your automated test suite. ### Timeline 1 to 2 weeks after contract start if all blockers are cleared. ### Onboarding to-dos Connect your Slack, Teams, or Discord account with QA Wolf. Send test environment credentials to your Account Executive, QA Engineering Lead, or Customer Success Manager. The environment needs to be stable and reachable before onboarding can begin. * Read: [Preparing an environment for QA Wolf test runs](/qawolf/Test-environments-2dc5b2a994fb800e8ce4fe36ee34764a) * Read: [Which environment should we run tests on?](/qawolf/Choose-the-right-CI-environment-to-execute-your-tests-29c5b2a994fb8031a3ddc1355eb4e7be) Identify a Champion (day-to-day) point of contact for QA Wolf who can clear blockers and make decisions about your test suite. ### Your QA team * Read: [Who will we be working with from QA Wolf?](/qawolf/Full-Service-FAQs-29c5b2a994fb8000b442fa8aed05c9e5#who-will-we-be-working-with-from-qa-wolf) ### Kick-off meeting **When** Within 5 business days of the contract start date. **Agenda** * Introductions. * Review coverage goals. * Set expectations for the process. * Align on milestones. **Attending** | From your team | From our team | | :------------------------------------------------------------------------------- | :----------------------- | | Point of contact during the sales process | Customer Success Manager | | Champion (day-to-day contact) who will be most hands-on working with us | QA Lead | | Any other team members you'd like to include (engineers, product managers, etc.) | | **Action items** | Your responsibilities | Our responsibilities | | :------------------------------------------------- | :---------------------------------------------- | | Connect a shared Slack, Teams, or Discord channel. | Learn about QA priorities and onboarding goals. | | Share your environment credentials. | | ### Product tour meeting **When** Within 5 business days of the contract start date. **Agenda** * Introductions. * Walk through your app. * Identify testing priorities. * Identify areas to avoid. **Attending** | From your team | From our team | | :----------------------------------------------------------- | :------------------ | | Product or engineering manager for each testable feature set | QA Engineering Lead | | | Test outliner | **Action items** | Your responsibilities | Our responsibilities | | :---------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------- | | Internally align on the product(s) that QA Wolf will be working with and what areas are the highest priority to build coverage for. | Record the product tour. | | | Take detailed notes for test plan creation. | | | Ask clarifying questions about priorities and product functionality. | ## Implementation Once we understand your application and testing priorities, we're ready to start outlining a Test Plan and coding up your automated test suite. ### Timeline * Planning phase: 2 to 4 weeks after the contract start date if all blockers are cleared. * Test creation phase: The next 2 to 3 months after the Test Plan is approved. ### Test planning to-dos 1. **Send over any existing test plans or internal priority lists.** Anything you have is great: Google Sheets, a TestRail matrix, or just a list of priority features. If you don't have anything, don't worry — we'll work with you to determine priorities and coverage areas. 2. **Export and send your existing E2E tests — if you have any.** We can convert that code to QA Wolf outlines and test code, or avoid covered product areas if you plan to continue running and maintaining them internally. ### First Test Plan review meeting **When** Within 15 business days of the contract start date. **Agenda** * Introductions. * Review the test plan. * Re-confirm priorities. * Provide feedback on the test plan's coverage areas. * Confirm the scope of work. **Attending** | From your team | From our team | | :----------------------------------------------------------- | :------------------ | | Product or engineering manager for each testable feature set | QA Engineering Lead | | | Test outliner | **Action items** | Your responsibilities | Our responsibilities | | :-------------------------------------------------------------------- | :----------------------------------------------------------------- | | Answer questions from the test outliners on Slack, Teams, or Discord. | Prepare detailed outlines for all tests within budget. | | | Field questions about the test plan and outlining process. | | | Respond to outline feedback and changes in your team's priorities. | ### Final Test Plan review **When** Within 20 business days of the contract start date. **Agenda** * Introductions. * Recap feedback from the first review. * Review changes/updates to the test plan. * Review the AAA framework. * Finalize the scope of work. **Attending** | From your team | From our team | | :----------------------------------------------------------- | :------------------ | | Product or engineering manager for each testable feature set | QA Engineering Lead | | | Test outliner | **Action items** | Your responsibilities | Our responsibilities | | :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------ | | Identify any remaining gaps or issues. | Convert all approved outlines to the [AAA framework](https://www.qawolf.com/blog/intro-to-aaa) in preparation for coding. | | Approve the scope of work. | Update outlines and priorities based on feedback from the first review. | ### Common test planning blockers * Your stakeholders aren't aligned on testing goals and priorities, which prevents us from finalizing a Test Plan. * Your team is unresponsive when QA Wolf reports possible bugs or asks for product clarification. * Your team is unavailable to review the test plan. ### Common test creation blockers * Your test environment is too unstable to run end-to-end tests reliably. Read: [Preparing an environment for QA Wolf test runs](/qawolf/Test-environments-2dc5b2a994fb800e8ce4fe36ee34764a). # How to integrate GitHub/GitHub Actions Source: https://docs.qawolf.com/qawolf/GitHub-GitHub-Actions-2a15b2a994fb800699f6dc99584915d0 Connect QA Wolf to GitHub Actions to automatically trigger test runs when you deploy code from your repository. If your repositories are hosted on GitHub.com or GitHub Enterprise Cloud, this is the recommended setup. The integration connects directly to your QA Wolf environments using GitHub Actions to notify QA Wolf of new deployments. If your team self-hosts GitHub Enterprise Server without internet access, or has disabled GitHub Actions by policy, use the [QA Wolf CI SDK](/qawolf/Integrate-with-the-QA-Wolf-SDK-2db5b2a994fb8043be61f06f025d9c63) instead. Make sure you have: * Access to your QA Wolf workspace. * Admin access to your GitHub repository. ## Enable the GitHub integration in QA Wolf Open the `Workspace name` dropdown in QA Wolf and click **Workspace Settings**. Choose **Integrations** and then click **Enable** next to **GitHub**. This opens GitHub, where you'll be prompted to sign in if you have not already done so. Your user cannot be a member of multiple GitHub orgs. Choose the repositories QA Wolf should access. Click **Install**. Once installed, QA Wolf can create PR comments. ### Find the QAWOLF\_API\_KEY Open the `Workspace name` dropdown in QA Wolf and click **Workspace Settings**. Choose **Integrations**. Generate your **QAWOLF\_API\_KEY** by clicking the icon to the right of **API Key** under **API Access**. ### Add the QAWOLF\_API\_KEY secret Return to GitHub, go to your repository's **Settings** tab, click **Secrets and variables**, and then **Actions**. Type `QAWOLF_API_KEY` in **Name** and paste your API key into **Secret**. Click **Add secret** to save. This key allows your workflow to authenticate with QA Wolf when the Action runs. ## Add the GitHub Actions workflow QA Wolf's official GitHub Action, [Notify QA Wolf on Deploy](https://github.com/marketplace/actions/notify-qa-wolf-on-deploy), notifies QA Wolf whenever a new deployment is ready for testing. This must be done for every repository you want connected to CI/CD. Open your repository on GitHub. Click the **Actions** tab. If you already have actions, click **New workflow**. Choose **Set up a workflow yourself**. Copy the example below into your workflow file (usually `.github/workflows/deploy.yml`). Replace the values in `deployment-type` and `deployment-url` with your actual values. To get the value for `deployment-type`, reach out to QA Wolf and we will provide this value for you. ```yaml theme={null} - name: Notify QA Wolf uses: qawolf/notify-qawolf-on-deploy-action@v1 with: qawolf-api-key: ${{ secrets.QAWOLF_API_KEY }} deployment-type: "staging" deployment-url: "https://staging.example.com" ``` ## Verify the integration Push a new commit to your main branch and wait for your deployment to complete. Open QA Wolf and confirm a new run appears under the expected environment. # How to integrate with GitLab Source: https://docs.qawolf.com/qawolf/GitLab-2a15b2a994fb805d9d14ff27726065ae Connect QA Wolf to GitLab CI/CD to automatically trigger test runs when you deploy code from your repository. If your repositories are hosted on GitLab.com or on a GitLab Enterprise Server instance with internet access, this is the recommended setup. The integration connects directly to your QA Wolf environments. If your GitLab instance does not allow outbound HTTPS connections (for example, an air-gapped GitLab Enterprise installation), use the QA Wolf CI SDK instead. Make sure you have: * Admin or Maintainer access to the GitLab project. * Access to your QA Wolf workspace. * At least one QA Wolf environment already configured. * A QA Wolf API key. ## Enable the GitLab integration in QA Wolf Open the `Workspace name` dropdown in QA Wolf and click **Workspace Settings**. Click **Integrations**, then click **Enable** next to **GitLab**. This opens GitLab, where you'll be prompted to sign in if you have not already done so. Follow the prompts to connect your GitLab account or group. You'll be asked to provide a **Group Access Token** with the **Maintainer** role and **API** scope. The Maintainer role is required because Developer tokens cannot create pipeline checks on protected branches. Once connected, QA Wolf can set commit statuses and link test runs to GitLab commits. ### Find the QAWOLF\_API\_KEY Open the `Workspace name` dropdown in QA Wolf and click **Workspace Settings**. Choose **Integrations** and then click the icon to the right of **API Key** under **API Access**. ### Add the QAWOLF\_API\_KEY secret Open your GitLab project. Go to **Settings → CI/CD → Variables**. Click **Add variable**. Name the variable `QAWOLF_API_KEY`, paste your API key, and save. You can also define additional variables, such as a deployment URL, if your pipeline uses them. ## Add the deploy notification to GitLab CI/CD QA Wolf provides a public API endpoint that your GitLab pipeline can call upon successful deployment. Add a job to your `.gitlab-ci.yml` that runs after your deploy step completes successfully. ```yml expandable theme={null} stages: - build - deploy - notify deploy: stage: deploy script: - ./deploy.sh environment: name: staging notify_qawolf: stage: notify script: - | curl -X POST "https://app.qawolf.com/api/webhooks/deploy_success" \ -H "Authorization: Bearer $QAWOLF_API_KEY" \ -H "Content-Type: application/json" \ -d "{ \"branch\": \"$CI_COMMIT_REF_NAME\", \"deployment_type\": \"staging\", \"deployment_url\": \"$QA_WOLF_DEPLOY_URL\", \"sha\": \"$CI_COMMIT_SHA\", \"hosting_service\": \"GitLab\" }" when: on_success needs: ["deploy"] ``` This job tells QA Wolf that a new deployment is ready for testing. Run your GitLab CI/CD pipeline to trigger a deployment. Wait for the pipeline to complete successfully. Open QA Wolf and confirm a new run appears under the expected environment. ## Verify the integration Push a new commit to your main branch and wait for your deployment to complete. Open QA Wolf and confirm a new run appears under the expected environment. ## Related * [webhooks/deploy\_success](/qawolf/deploy-success) * [REST API](/qawolf/rest-overview) # Glossary Source: https://docs.qawolf.com/qawolf/Glossary-29c5b2a994fb806d93f2ce4554a2fcfc Definitions of key QA Wolf terms including workspaces, environments, groups, flows, runs, coverage outlines, and coverage maps. ## Workspace A **workspace** is the top-level container in the QA Wolf platform. All environments, groups, and related settings live inside a workspace. Use workspaces for separate applications, such as the web, iOS, and Android versions of your product. Or when different business units are responsible for completely separate test suites. ## Environment Under each workspace is an **environment**, which corresponds to a specific version of your app, such as development, staging, or production. Each environment can be [configured](/qawolf/Environment-settings-2cd5b2a994fb8014ba8bd7aa25f3ed0f) with separate variables (e.g., base URL, users, etc.), concurrency rates, and other settings relevant to the version of the application you're testing. Flows and tests are unique for each environment and not shared between environments. ## Flow A **flow** is a sequence of tests that validates a user journey in your application. Flows are the primary unit of end-to-end testing in QA Wolf. Flows can be run individually, scheduled, and tagged. ## Test A **test** verifies a specific behavior or step in your application. Multiple tests combine to form a flow that validates a complete user journey. **Example: QA Wolf application hierarchy** | Item | Example | | :---------- | :---------------------- | | Workspace | Wolf Corp | | Environment | Staging | | Folder | accounts | | Flow | happy-path-ios.flow\.js | ## Coverage outline A **coverage outline** is the set of groups and flow stubs that define what needs to be tested in an environment. Outlining is the act of producing one: you point the agent at your application, it explores, and it proposes the groups and flows that represent the main areas and user journeys. An outline records scope — it does not contain test code until the flows are created. ## Coverage map The **coverage map** is the visual representation of a coverage outline. It shows the flows in an environment and how they are organized into groups, so you can see the shape of your coverage and find gaps in it. ## Run A [run](/qawolf/Run-a-flow-or-suite-2a25b2a994fb8014b164d7ad2efd5ddel) is an execution of one or more flows. Each flow is attempted up to three times per run if it fails. Runs can be started manually or invoked automatically by a schedule. The status and results of every flow in a run are recorded and reported back. ## Attempt Within a run, a flow may be attempted multiple times. As a tester, you will occasionally have flows that flake — attempting to run a failed flow multiple times ensures that the failure is not a temporary fluke. QA Wolf will attempt to run a flow up to three times. Each attempt has its own video, logs, and results, allowing you to isolate issues precisely. ## Schedule A schedule defines when and how often runs occur automatically. Scheduling is limited to flows. Individual tests can be run manually for debugging, but they cannot be scheduled independently. Use schedules to execute flows at specific times (midnight) or at recurring intervals (every hour), or when a build is deployed. # Integrate a mobile build with GitHub Actions Source: https://docs.qawolf.com/qawolf/Integrate-a-mobile-build-with-GitHub-Actions-2db5b2a994fb8027a901d4238b02fb8f Use the QA Wolf GitHub Actions to upload mobile build artifacts and trigger test runs directly from your existing workflows. ## When to use GitHub Actions This guide is for teams that use **GitHub Actions** as their CI system and want a simple, declarative way to upload mobile builds and trigger test runs without writing custom scripts. Use GitHub Actions if your repository already builds mobile artifacts in a GitHub Actions workflow and you prefer using prebuilt actions over maintaining Node.js scripts or Fastlane lanes. This guide assumes your mobile builds already run in GitHub Actions and produce a build artifact. ## Before you begin 1. Make sure your GitHub Actions workflow produces a mobile build artifact (APK, AAB, or IPA). 2. Artifact naming conventions are defined for your environments. See [Artifact naming conventions](#artifact-naming-conventions) below. Before mobile test runs can execute, QA Wolf must enable mobile triggers for your workspace. QA Wolf will handle this and may ask you for: * Which environments you want to test. * Whether PR testing is enabled. * The artifact naming conventions you are using. * The upload and trigger method you chose. Until this step is complete, CI jobs can upload artifacts and send deployment notifications, but mobile test runs will not start automatically. ## How GitHub Actions works with QA Wolf The GitHub Actions integration uses two QA Wolf-provided actions: Upload a mobile build artifact to QA Wolf. Notify QA Wolf of a deployment event to trigger a test run. You can upload builds without triggering runs, which is useful during initial setup or validation. You provide the [artifact basename](#artifact-naming-conventions) when uploading. QA Wolf applies the file extension automatically based on the uploaded file. ## Add QA Wolf actions to your workflow Add the QA Wolf actions to an existing GitHub Actions workflow that builds your mobile app. The workflow must run after the build artifact has been created. ### Find the QAWOLF\_API\_KEY Open the `Workspace name` dropdown in QA Wolf and click **Workspace Settings**. Choose **Integrations**. Generate your **QAWOLF\_API\_KEY** by clicking the icon to the right of **API Key** under **API Access**. ### Add the QAWOLF\_API\_KEY secret Open your GitHub repository and go to **Settings**. Select **Secrets and variables → Actions**. Add a new repository secret named `QAWOLF_API_KEY` and paste your API key. ## Artifact naming conventions Mobile build artifacts must follow consistent naming conventions so QA Wolf can correctly associate each build with the right environment and make failures easier to diagnose. The artifact name is used to identify: * Which environment the build belongs to. * Whether the build is tied to a pull request. * Which build was used for a given test run. ### Static environments Static environments are long-lived environments such as staging or release environments. **Format** ```text theme={null} - ``` **Example** ```text theme={null} app-staging ``` Use the same basename every time a build is generated for the same environment. ### PR (ephemeral) environments PR environments are short-lived and tied to a specific pull request. These are only relevant if PR testing is enabled. **Format** ```text theme={null} ---pr ``` **Example** ```text theme={null} app-myorg-myrepo-pr123 ``` Including the organization, repository, and pull request number ensures each build can be traced back to the correct change and environment. QA Wolf applies the file extension (.apk, .aab, or .ipa) automatically based on the uploaded artifact. You only need to provide the basename. ## Upload a mobile build artifact After your workflow produces a mobile build artifact, upload it to QA Wolf using the upload action. ```yaml theme={null} - name: Upload mobile build to QA Wolf id: upload-run-input uses: qawolf/upload-run-inputs-executable-action@v1 with: qawolf-api-key: ${{ secrets.QAWOLF_API_KEY }} input-file-path: ./path/to/build.apk executable-file-basename: app-staging ``` If this step completes successfully, the artifact is uploaded and available for test runs. ```yaml expandable highlight={1-25} theme={null} name: Deploy and Notify QA Wolf on: pull_request jobs: ... notify: needs: deploy-preview-environment name: Trigger QA Wolf PR testing runs-on: ubuntu-latest steps: ... # Upload the run input file - name: Upload Run Input id: upload-run-inputs-executable uses: qawolf/upload-run-inputs-executable-action@v1 with: qawolf-api-key: "${{ secrets.QAWOLF_API_KEY }}" input-file-path: "path/to/file.apk" - name: Notify QA Wolf of deployment uses: qawolf/notify-qawolf-on-deploy-action@v1 env: ... # Use the output in the RUN_INPUT_PATH environmental variable RUN_INPUT_PATH: "${{ steps.upload-run-inputs-executable.outputs.destination-file-path }}" ... with: ... ``` ## Trigger a test run After uploading the artifact, notify QA Wolf that a new deployment is ready for testing. ```yaml theme={null} - name: Notify QA Wolf of deployment uses: qawolf/notify-qawolf-on-deploy-action@v1 with: qawolf-api-key: ${{ secrets.QAWOLF_API_KEY }} deployment-type: android_app variables: > { "RUN_INPUT_PATH": "/home/wolf/run-inputs-executables/${{ steps.upload-run-input.outputs.destination-file-path }}" } ``` The deployment type and environment key must match the values configured by QA Wolf for your workspace. If mobile triggers have not yet been enabled, this step will complete without starting a test run. ## Verify the integration Run the GitHub Actions workflow. Verify that the artifact upload step completes successfully. Confirm that the deployment notification step runs without errors. Once mobile triggers are enabled, check the **Runs** tab for the triggered test run. ## Troubleshooting and common issues * **If uploads succeed but no runs start:** Mobile triggers may not yet be enabled. Contact QA Wolf to complete platform configuration. * **If the artifact is not found during execution:** Verify that the artifact basename matches your naming conventions and that the destination file path from the upload step is used to trigger the run. * **If you see authentication errors:** Verify that **QAWOLF\_API\_KEY** is configured correctly as a GitHub Actions secret. * **If the workflow fails before the QA Wolf steps run:** Verify that the mobile build step completes successfully and produces the expected artifact. # How to integrate a non-native bug tracker with webhooks Source: https://docs.qawolf.com/qawolf/Integrate-with-webhooks-29c5b2a994fb818d9682eb32ff11feff If we don't support your bug tracker through a native integration, you can still use webhooks to integrate. We know that each implementation will vary. Please reach out and let us know if you need any support. ### Set up the integration Open the `Workspace name` dropdown and click **Workspace settings**. Navigate to **Integrations**. Click **Enable** next to Webhook. Click **Connect** in the modal. Define a webhook title. Define the webhook URL and enable "Notify when bug report is opened." This is the location of the webhook used by your bug tracker. When an issue is created, your endpoint will receive `POST` requests with a JSON body (`Content-Type: application/json`). Here's the expected structure and an example payload. #### Expected JSON Body Fields The primary data is in the request body: * `id` (`string`): Unique bug report ID. * `title` (`string`): Bug report title. * `description` (`string`): Detailed description (may contain newlines/links). * `url` (`string`): Link to view the bug report in QA Wolf. * `event` (`string`): `bug-report-opened`. #### Example Payload This example shows the full request structure your endpoint will receive: ```json JSON theme={null} { "id": "", "title": "Bug Report Title", "description": "🐞\n\nBug report description:\nYou can see it here.\n\nBug report:\nhttps://app.qawolf.com/test-client/bug-reports/...\n\nAffected workflows:\n\n\nIf you are aware of this bug you can set the priority to low which will prevent it from causing a run failure.", "url": "https://app.qawolf.com/bug-reports/...", "event": "bug-report-opened" } ``` #### Authentication If username and password are set, those will be used for basic HTTP authorization when performing the `POST` request against your webhook. Enable the "Notify when bug report is closed" toggle: Make your webhook respond to the `POST` request with a payload containing these fields: * `externalId` (`string`): ID of the issue for the QA Wolf bug report in your issue tracker. It will be used to identify issues that will need to be updated in the future. * `humanId` (`string`): Human-friendly issue ID that will be used on QA Wolf UI, Slack messages and other integrations. * `url` (`string`): Link to the issue. **When a bug report is closed**, your endpoint will receive `POST` requests with a JSON body (`Content-Type: application/json`). Here's the expected structure and an example payload. #### Expected JSON Body Fields The primary data is in the request body: * `externalId` (`string`): The ID of the issue associated with the closed bug report. * `event` (`string`): `bug-report-closed`. # Integrating with Microsoft Teams Source: https://docs.qawolf.com/qawolf/Integrating-with-microsoft-teams Set up Microsoft Teams to receive QA Wolf notifications by configuring external access and shared channels. ## Completion checklist In the [Teams Admin Center](https://admin.teams.microsoft.com/): Teams policies > Join external shared channels to On Users > External Access > Ensure QA Wolf is not blocked In the [Microsoft Entra admin center](https://entra.microsoft.com/) (**as a Security Administrator**): Add QA Wolf's tenant ID - `3d41cc11-5772-457c-aad2-16b3924b22d0` Verify access settings do not restrict access to or from QA Wolf, or adjust accordingly Share your Microsoft Entra tenant ID with QA Wolf ### Enable shared channels in Teams Shared channels are **enabled by default** in Teams. QA Wolf provides a shared channel for your Teams integration. To use it, your organization must have shared channels enabled. In the [Microsoft Teams admin center](https://admin.teams.microsoft.com/), expand **Teams** in the left navigation bar and choose **Teams policies**. Click the policy you want to confirm or change. Under *Shared channels settings*, turn on **Join external shared channels** to allow collaboration with other orgs. Click **Apply** to save your changes. ### Enable external access You must enable external access so QA Wolf team members can join your **Teams meetings** and your team can see QA Wolf's presence status (online/offline/away) in shared channels. In the [Teams admin center](https://admin.teams.microsoft.com/), expand **Users** in the left navigation bar and choose **External access**. Choose the appropriate option for allowing **Teams and Skype for Business users in external organizations**. **Do not** select **Block all external domains** — this setting prevents the integration from working. Allowing **qawolf.com** under **Allow only specific external domains** will ensure QA Wolf team members can participate while all other external domains remain blocked. ### Enable cross-tenant access in Microsoft Entra To use a shared channel with QA Wolf, you must allow our Microsoft Entra tenant by adding QA Wolf as an allowed organization using our Microsoft Entra tenant ID. As a **Security Administrator**, sign in to the [Microsoft Entra admin center](https://entra.microsoft.com/). Choose **External Identities** from the left navigation bar, then choose **Cross-tenant access settings**. On the **Organizational settings** tab, click **Add organization**. In the **Add Organization** drawer on the right, type QA Wolf's tenant ID: `3d41cc11-5772-457c-aad2-16b3924b22d0` and click **Add** at the bottom. If your default access settings restrict access, update your inbound and outbound B2B direct connect settings to allow QA Wolf. Still under Organizational Settings, you should see QA Wolf listed under **organizations found**. #### Check access Click on the Configured link underneath Inbound access or Outbound access. Click the B2B direct connect tab. Select the Customize settings radio button. On the **External users and groups** tab: * For **Access status**, select *Allow access* * For **Applies to**, select *All QA Wolf users and groups* After updating these settings, verify that your default access configuration no longer blocks access to or from QA Wolf. Shared channel collaboration will not work if the default settings continue to block external tenants. We use shared channels so customers can access them without switching tenants, and B2B Direct Connect must be used to invite or join them. **B2B Collaboration** will not work (see [*MS doc*](https://learn.microsoft.com/en-us/microsoft-365/solutions/collaborate-teams-direct-connect?view=o365-worldwide) for details). | B2B Collaboration | B2B Direct Connect | | ------------------------------- | ----------------------------------------- | | Applies to guest accounts | Enables direct shared channel invitations | | Requires tenant switching | Seamless access from own tenant | | Only supports standard channels | Fully supports shared channels | #### Complete setup We need your **Microsoft Entra tenant ID** and **domain name** so we can configure cross-tenant access on our side. Customers can locate their Tenant ID and domain on the [Microsoft Entra dashboard](https://entra.microsoft.com/#home). # Reviewing run results Source: https://docs.qawolf.com/qawolf/Interpret-run-results-2d55b2a994fb80199754e4b2d3ce0318 Navigate the Runs tab to review pass rates, failure counts, bug reports, and maintenance status for each run. ## Overview For each run, the list shows: * The name of the schedule that triggered the run * — the number of fully passing attempts over total attempts * — the number of flows still requiring investigation * — the number of open bug reports linked to flows in the run * — the number of open maintenance reports linked to flows in the run * Total wall clock time of the run, including all attempts Runs ## Identify environments that need attention The icon next to each environment name indicates its current status. | Status | Meaning | | :----------------------------------------- | :-------------------------------------------------------------------------- | | Ready | All flows passing | | Needs investigation | One or more flows failed, or a medium or low priority bug has been reported | | Failed | A high severity bug has been reported | You can see environment status in two places: * On the **Runs** tab, next to each environment name in the list on the left * On the **Flows** tab, in the environment name dropdown ## Review a run From the **Runs** tab, select the environment you want to investigate. Runs are grouped into three categories: * **Investigating** — flows in the run failed and have not been marked *Do Not Investigate* * **Completed** — all flows have passed or been marked *Do Not Investigate* * **Canceled** — the run was canceled mid-execution by a subsequent run matching the same branch and environment Click the run you want to investigate. ## Sort and filter runs The default view shows past runs sorted by date in descending order. Click the icon to filter by: * **Created date** — *1 week ago*, *1 month ago*, or *1 year ago* * **Trigger** — *Hourly* or *Daily* # How to integrate with Jira Source: https://docs.qawolf.com/qawolf/Jira-29c5b2a994fb81d18a87f23808479de0 Connect QA Wolf to Jira to automatically create, sync, and close issue tickets from QA Wolf bug reports. ## Overview * **Automatic issue handling.** Creates Jira issues automatically when new bug reports are generated in QA Wolf, keeping Jira up to date without manual entry. Context added in Jira is not written back to QA Wolf. * **Synchronization.** Closing a bug report in QA Wolf will close the linked Jira ticket. QA Wolf also performs regular bug revalidations to ensure accuracy. Closing a Jira ticket should automatically update QA Wolf. Contact us if that doesn't happen so we can help you troubleshoot. * **Test creation requests.** Add a **QA Wolf** label to a Jira ticket to automatically create a coverage request in QA Wolf. * **Custom configuration.** Specify the project, assignee, and issue status. Currently, each integration setup supports one project, one assignee, and one status type. * **Custom fields.** Add custom Jira fields to bug reports if your Jira project uses them. Custom fields apply globally across all tickets created through QA Wolf, not per workflow. **Full Service Customers:** Need help setting up your JIRA integration? The QA Wolf team can configure this integration on your behalf. To get started, invite us as a user in your JIRA workspace using the following email convention: qa+\[yourcompanyname]@qawolf.com (e.g. [qa+acme@qawolf.com](mailto:qa+acme@qawolf.com)). ## Set up the integration Open the `Workspace name` dropdown and click **Workspace settings**. Navigate to **Integrations**. Click **Enable** next to Jira. The **Sync bug reports to Jira** modal appears. Click **Connect**. Select the Jira team you want to link to your QA Wolf team and click **Accept** to authorize the connection. Choose your Jira **Project**, **Issue Type**, and **Assignee**, then enable: * **Create bug issue report** * **Close bug issue report** This ensures bugs are created and closed automatically across both systems. If there are any custom fields you need us to configure, such as the issue description, please reach out and let us know. We can configure those for you. # How to integrate with Linear Source: https://docs.qawolf.com/qawolf/Linear-29c5b2a994fb81ce85ded6d005d5e7ab Connect QA Wolf to Linear to automatically create and sync bug report issues with your Linear team. Open the `Workspace name` dropdown and click **Workspace settings**. Navigate to **Integrations**. Click **Enable** next to **Linear**. A modal opens. Click **Connect**. Enter the **Team name** exactly as in Linear. Team name is case sensitive. Enable the **Create bug report issue** toggle and optionally set **Label UUID** and **Assignee UUID** to set a default label and assignee for newly created issues. # File, review & edit bug reports Source: https://docs.qawolf.com/qawolf/Manage-bug-reports-2c75b2a994fb80308ddcc7cc60703c2f Create, review, and resolve bug reports in QA Wolf to track confirmed product defects linked to failing flows. ## Overview Bug reports track confirmed product defects, specifically unexpected application behavior rather than test or setup issues. Bug reports are automatically associated with flows when opened from the Investigation view. Multiple flows can be linked to the same bug report when they expose the same issue. A bug report automatically closes once all associated flows are passing. Once the test issues are fixed, you can attempt to clear the bug report by running all associated flows directly from the report. Any flows that pass are removed from the report and resume running according to their schedules. The bug report remains open until all associated flows have passing runs. **Best practices** * Use bug reports to track and resolve underlying product issues. * Allow flows with known bugs to continue running to detect when the issue is fixed. * Avoid quarantining flows for known bugs unless execution is causing operational issues. ## Get to know the bug report tab You can find all bug reports for your workspace in the **Bugs** tab. This page lists both open and closed reports and allows you to create new ones or open existing ones for editing. * Use the **Open** and **Closed** tabs to switch between active and resolved reports. Bug Report List * Use the **Search** bar to filter by report name. Use the icon to filter by report **Priority** or **Creator is me**. ## File a bug report In the Investigation view, select the failing flow. **Under Diagnose**, click **Report a bug**. You can also find **Diagnose** by hovering over the failing flow to show the icon, then clicking it to show the icon. **Diagnose** will appear below the Flow list. A drawer appears, displaying a pre-populated form in QA Wolf's standard bug-report format. Bug Report Drawer Click the **Name** field to open a dropdown of existing bug reports. As you type, the list autocompletes to help you find a match. a. If you need a new report, type its name and press **Enter**, or choose the top option in the dropdown. After selecting a name, fill out the remaining form. The form contains the following fields: a. **Description:** Provide a clear summary of the issue. This field includes a built-in template to help you capture the most important details—such as expected behavior, actual behavior, and steps to reproduce. You can also include a link to a cloud-hosted video (e.g., Loom) in the **Video** field to illustrate the problem. b. **Priority:** Set the bug's urgency. **Urgent** and **High** represent blocking priorities and will affect the overall environment status. c. **Environment (read-only):** Indicates the environment where you observed the bug. d. **Affected flows (read-only):** Lists the flows impacted by this bug. **Flows with open bugs are still included in scheduled runs.** * If any bug associated with a flow has a blocking **Priority** (*Urgent* or *High*) and at least one flow in the run fails all three attempts, the run ends in a *Failed* status. The environment's status will appear with the icon on both the **Flows** and **Runs** tabs. * If the flow has only non-blocking bugs, and at least one flow in the run fails all three attempts, the run ends as *Needs investigation*. The environment's status will appear with the icon on both the **Flows** and **Runs** tabs. You can also file bug reports from the **Bugs** tab by clicking **Report bug** to open the form for filing a new bug. However, bugs you open from this method will not be associated with any flows. To associate this bug with flows, you must follow the **Diagnose** steps above. ## Review or edit an existing bug report * From the **Bugs** tab, find the bug report you'd like to open and click on it. * The only field you can change here is the **Priority** field. All other fields are read-only. ## Clear bug reports A bug report should close automatically on the next scheduled run once you apply the fix. However, you can also manually run all flows associated with a given bug; if they pass, the bug report will close automatically. Open the **Bugs** tab and click the report you'd like to close. Click **Run all flows**. This opens the **Runs** tab in a separate browser tab and starts a manual run of all flows associated with the bug report. Any flows that pass will be removed from the bug report. If all flows pass, the bug report will close automatically. ## Update the diagnosis of a test failure Sometimes, a run will still fail after an application fix is deployed. This can happen when the original issue masked a different problem, or when the fix changes behavior in a way that affects the test. If this happens after a developer ships a fix: Open the failed run under **Needs investigation** to enter the **Investigation view**. Re-run the failing flow to confirm whether the failure still occurs. Review the failure evidence: * If the flow already has a bug filed and the failure points to a new application issue upstream of the existing one, click **Update diagnosis**, then select **Report a bug** to file a new bug report. * If the flow already has a bug filed but the failure now indicates the test needs to be updated, click **Update diagnosis**, then select **Report as needing maintenance**. This keeps diagnoses accurate as failures evolve, without tying the new failure too closely to the original issue. # Managing maintenance reports Source: https://docs.qawolf.com/qawolf/Manage-maintenance-reports-2d55b2a994fb80728909f48b1023f7fa File maintenance reports to quarantine failing flows caused by test issues, and resume them after fixes are applied. Maintenance reports are used when a failure is caused by incorrect, outdated, or fragile test logic rather than an issue in the application under test. Maintenance reports cannot be created directly from the Maintenance tab. They are created from the Investigation view when reviewing a failing flow. Minor test fixes should be made quickly when possible. File a maintenance report when a fix will not be completed before the next deployment or scheduled run, or when the change requires more involved updates to the test, such as refactoring. Maintenance reports are linked to one or more flows. While a maintenance report is open, those flows are skipped in scheduled runs. When the flows pass in a manual run, the maintenance report automatically closes, and the flows are included in schedules again. ## Creating a maintenance report Use a maintenance report for any failing flow that will take more than about 15 minutes to fix (a guideline, not a rule), and quarantine the flow. From the **Runs** tab, select the environment from the **Environments** list. In the center panel, select a run from **Investigating**. In the Investigation view, select the failing flow. In the center panel, click **Diagnose**. Click **Report as needing maintenance** to open the drawer for creating a maintenance report. Click the **Name** field to open a dropdown of existing maintenance reports. As you type, the list autocompletes to help you find a match. Selecting an existing report will add your flow to that report in the **Assign flows** field below. a. If you need a new report, type its name and press **Enter**, or choose the top option in the dropdown. After selecting a name, fill out the remaining form. a. **Description:** Provide a short explanation of why the flow(s) need maintenance. Use this to outline the issue, what's outdated or broken, and any relevant context. b. **Effort:** Estimate the effort required to update or repair the flow(s). c. **Priority:** Set the urgency for maintenance. Flows associated with higher priority reports appear more prominently in the environment status. d. **Reason for Maintenance:** Select the appropriate reason from the predefined list—these standard options help categorize and report on maintenance work consistently. e. **Environment (read-only):** This displays the environment affected by the maintenance. The system automatically populates the field. f. **Assigned Flows (read-only):** This displays the flows included in the maintenance report. g. New reports automatically include only the flow where you initiated the report. Submitting this change removes the flow from all runs. ## Viewing a maintenance report * To view all maintenance reports, open the **Maintenance** tab. * The tab includes two subtabs: 1. Filter by readiness status. 2. Search by report name. ## Closing a maintenance report A maintenance report must be closed manually. When you fix a test with an open maintenance report: Open the **Maintenance** tab, then click the maintenance report associated with the flow you fixed. Click **Run all flows**. This opens the Runs tab in a separate browser tab and starts a manual run of all flows associated with the maintenance report. Any flows that pass will be removed from the maintenance report. If all flows pass, the maintenance report will close automatically. # Create a coverage outline Source: https://docs.qawolf.com/qawolf/Map-your-app-s-workflows-2a45b2a994fb80ca8a87c8fdc2a443fc Use the outlining agent to build and maintain a coverage outline of your application's user journeys. A coverage outline defines the scope required to test an application end-to-end. Without visibility into an application's surface area — key user paths, important states, and system boundaries — it's difficult to know whether coverage is complete or where it isn't. The outlining agent builds that outline by exploring your application and generating the groups and flow stubs that represent its main areas and user journeys. The result is rendered as a coverage map, the visual view of what you cover. As features change or new ones are added, you can run the agent again to extend the outline.