Automated Testing

Automated testing is the application of software tools to automate a human-driven manual process of reviewing and validating a software product. Most modern agile and DevOps software projects now include automated testing from inception - and Altus is no different.

To assist with conducting Altus customisations via modern Agile processes, we include access to the Automated Test Suite for Altus. These tests use a variety of technologies including reqnroll and Playwright to automate the validation of Altus environments.

How to Clone the Repository and Setup Visual Studio

Pre-requisite: Visual Studio 2022 is installed with .NET desktop development.

Steps:

1. Open Visual Studio.

2. Sign In using Microsoft account.

3. Click on "Clone a repository".

4. Enter the following details then click on "Clone":

   1. Repository location: "https://ipmo.visualstudio.com/Sensei.Kaizen.AutomationTests/\_git/Altus.Regression.Suite".

      Note

      This step can only currently be done by Partner consultants. We are considering moving this to a public repository over time.

   2. Path: Desired folder location.

   3. Login using Partner account (user should have access to the repo).

   4. You should see "Altus.Regression.Suite" solution in the Solution Explorer.

5. (Optional) Add reqnroll as an extension (Note: You only need to do this once for your Visual Studio instance):

   1. Navigate to Extensions menu > Manage Extensions.
   2. Click on "Online" tree view on the left-hand navigation pane.
   3. Type "reqnroll" in the Search box on the right-hand navigation pane.
   4. Select "Reqnroll for Visual Studio 2022" in the search results.
   5. Click on Download.
      
      
   6. After the download completes, close Visual Studio.
   7. The VSIX installer pop-up will appear, click on "Modify".
      
   8. Close VSIX installer once the process is completed.
      

6. Re-open Visual Studio and add an environment config file:

   1. Open the user secrets in Visual Studio of the project.

   2. Update the config file using the below template then save changes. See reference document: Setting up credentials in Key Vault

      {    
          "Environment": "<EnvironmentName>",
          "Headless": "false",
          "Browser": "msedge",
          "Tracing": "false",
          "<EnvironmentName>": {
              "UseKeyVault": "<yes/no>",
              "KeyVaultUri": "<keyVaultUri>", //required if UseKeyVault is yes
              "KeyVaultClientId": "<AzureAppClientId>", //required if UseKeyVault is yes
              "KeyVaultSecret": "<AzureAppSecret>", //required if UseKeyVault is yes
              "KeyVaultTenantId": "<AzureTenantID>", //required if UseKeyVault is yes
              "TargetURL": "https://<TargetEnvironment>.dynamics.com/",
              "AuthUsername": "<UserName>", //required if UseKeyVault is no
              "BaseAuthPassword": "<PasswordInBase64>", //required if UseKeyVault is no
          }
      }
      

      Sample:
      

7. If not displayed, open the Test Explorer from the View menu.
   

Parameter Change to Run the Tests with the Browser Open

By default, the tests will run in headless mode, meaning you will not see the browser as the tests are being executed. If you want to run the tests in headful mode (with the browser open), follow the steps in this section. Otherwise, skip to the next section Running the Tests.

1. Open the user secrets in Visual Studio of the project.
2. Set Headless parameter to false.

Running the Tests

1. Build the solution via Build > Build Solution. Once successful, the tests will appear in the Test Explorer.
2. Expand on the arrows on the Test Explorer pane to display the tests.
3. Right-click on a test (you can select multiple tests by holding shift or control on the keyboard) and then click "Run".
4. The test will start to be executed indicated by the "running" icon on the test. Note: If the tests are run in headful mode, the browser will close after logging in to Altus. This is expected as the tests are using a global sign in method. A new browser will open with a signed-in state to run the actual test.
5. Once the test run is complete, passed tests will have a green tick indicator. You can also see the run results on the Tests section of the Output pane. If the Output pane is not open, click View menu > Output. Select Tests from the "Show output from:" dropdown.
6. To check test screenshots, navigate to the Screenshots folder of your project folder. The screenshot file name will be the same as the test name, prefixed with a timestamp and a "pass" or "FAIL" as appropriate.

Generating Reports

The results will be displayed in Test Explorer. Including links to pictures.

If Tracing is enabled in the configuration a Trace zip package is created by the tests in the output folder \bin\Debug\net8.0-windows8.0\traces. The Trace can bests be viewed with the Playwright Trace Viewer.s

Setting up Credentials in Azure Key Vault Secrets

Pre-requisite

User would need to have an Azure role assignment to be able to create Key Vaults.

Part I: Creating an Azure Key Vault

1. Login to Azure Portal.
2. Go to Key Vaults by searching for "key vaults" on the search bar and clicking on the search result.
3. Click Create.
4. Populate the required details on the Basics tab. Once done, click the Tags tab:
   1. Project details
      • Subscription: "your subscription"
      • Resource group: "your resource group"
   2. Instance details
      • Key vault name: "name of your key vault"
      • Region: "your region"
      • Pricing tier: Standard
   3. Recovery options – you can update this if you want but using the default values is fine.
5. Add a tag pair (in this key vault example we have Name: Environment and Value: Development). Once done, click Review + create.
6. Click Create.
7. Once the deployment is complete, click on Go to resource.
8. Take note of the Vault URI and the Directory ID. You will save these values in the config file for the parameter KeyVaultUri and KeyVaultTenantID.

Part II: Create Key Vault Secrets for your username and password

1. Click Secrets on the left pane and then click Generate/Import.
2. Populate the required details and click Create:
   • Name: AuthUsername
   • Secret value: "your username"
3. Create another secret for your password with the details:
   • Name: AuthPassword
   • Secret value: "your password"
4. You should have 2 entries on your Key Vault Secrets once done.

Part III: Give an AAD Application permission to access Key Vault Secrets

Pre-requisites

• An AAD Application has already been created to be used to access the Key Vault. We will use "Automation – Selenium Tests"
• A client secret has been generated for the AAD Application. Request this information from the AAD Application owner

1. Click Access policies on the left pane and then click Create.
2. On the Permissions tab, tick Get and List under the Secret permissions column and click Next.
3. On the Principal tab’s search bar, search for the AAD application you want to give access to your Key Vault.
4. Select the AAD application and click Next.
5. Click Next on the Application (optional) tab.
6. Click Create.
7. You should see the application appear on the Access policies of your key vault.

Part IV: Using your Key Vault for the automation tests

Update the Key Vault related parameters in your config.json1 file:

• KeyVaultUri – from Part I, Step 8
• KeyVaultTenantID - from Part I, Step 8
• KeyVaultClientId – request this from your AAD Application owner
• KeyVaultSecret – request this from your AAD Application owner

"UseKeyVault": "yes",
"KeyVaultUri": "<keyVaultUri>",
"KeyVaultClientId": "<AADAppClientId>",
"KeyVaultSecret": "<AADAppSecret>",
"KeyVaultTenantID": "<AADTenantID>