Xray

​

Overview

• Create test executions and link them to Xray test plans or test cases
• Sync test results (e.g., passed, failed, skipped) to Jira/Xray
• Update Jira issues with test execution results for better traceability

​

How It Works

​

What Gets Synced

• Test executions: Created under a specified Jira/Xray project
• Test case results: Passed, failed, skipped, etc.
• Metadata: Execution start/end times, environment info, and optionally test evidence (e.g., logs, screenshots)

​

Requirements

​

Xray/Jira Configuration

• ✅ Xray installed in your Jira Cloud instance
• ✅ A Jira project configured for use with Xray
• ✅ API access enabled
  • Requires Admin access to generate and authorize API credentials

​

Authentication

• Client ID & Client Secret (for the Xray Cloud API)
  • These are generated in the Xray API settings via the Xray Developer Portal
• Jira project key or URL where test executions should be created

​

Setup Instructions (Step-by-Step)

​

1. Enable API Access and Generate Credentials

​

2. Identify Target Project in Jira

• Choose or create a Jira project with Xray test cases and plans already configured.
• Share the project key (e.g., QA, TESTPROJ) and/or base project URL.

​

3. Configure Xray Test Cases

• Ensure all automated test cases are already created in Xray, or can be matched via identifiers (e.g., test keys like TEST-123).
• Optionally organize your tests into test sets or plans.

​

4. Configure custom Xray test statuses

• Add custom statuses for more accurate test results (see below for more details)
  • Xray documentation on how to add statuses as test run custom fields
• Custom statuses to add:
  • SKIPPED
  • MAINTENANCE
  • EXISTING ISSUE

​

5. Share Credentials Securely

• Client ID
• Client Secret
• Jira project key
• (Optional) default test plan to link executions

​

5. Trigger and Verify

• Run a sample test suite and verify a Test Execution appears in Xray.
• Validate that test cases are correctly mapped and statuses appear as expected.

​

Frequently Asked Questions (FAQ)

​

Known Limitations

• ✅ Cloud-hosted Jira/Xray only (Data Center and Server not currently supported)
• ⚠️ Only one project supported per integration
• ⚠️ Test cases must pre-exist in Xray
• ⚠️ Custom fields are not supported unless explicitly configured

​

Custom Status Configuration

• The relationship between QAW and Xray tests is important to consider in order understand how QAW test results will sync with your Xray test runs.
• Xray test runs have the following default statuses available:
  • PASSED
    • Passing test
  • FAILED
    • Bugged test
  • TODO
    • Not yet synced/executed
  • EXECUTING
    • Not typically used in our current integration but can be used as a substitute for a test in maintenance (or another status) upon request.
• Our current integration syncs the results of individual QAW tests within flows to individual test runs.
  • I.e. for a bugged QAW flow, QAW Test 1 will get a status of PASSED, QAW Test 2, QAW Test 3, and QAW Test 4 will get a status of FAILED.
    
  • An added custom status of SKIPPED is useful to differentiate between failed tests and tests that didn’t run due to a preceding test failing. I.e. QAW Test 3 and QAW Test 4 would get a status of SKIPPED.
• The custom status of MAINTENANCE is to indicate when a flow is placed under maintenance by your QAW team.
• EXISTING ISSUE is used to indicate that a previously reported bug is still present.
  • We have a custom implementation to automatically close highly prioritized bug reports found in previous test executions on request.

​

Support