Regression testing for mobile apps can be tough. The wide array of phones and OS versions makes testing for mobile applications a time consuming and tedious task. Luckily, tools like BrowserStack and Appium exist to make our lives a bit easier. App Automate is a product made by BrowserStack and uses Appium as a test driver. It offers a wide array of native phones with the supported OS'es through the App Live product.
This post will cover the following subjects:
- Automatic upload to App Automate from Visual Studio AppCenter.
- Set up a simple app test suite, running in App Automate.
Uploading app from Visual Studio AppCenter to BrowserStack Appium
By looking at the BrowserStack documentation, we notice that BrowserStack has a REST API that allows upload from a public location. We see the following cURL command:
curl -u "user:topsecret" -X POST "https://api-cloud.browserstack.com/app-automate/upload" -F "data={\"url\": \"https://www.browserstack.com/app-automate/sample-apps/android/WikipediaSample.apk\"}"
Did you know that AppCenter has a REST API as well? Make sure that you issue an API key from AppCenter and you're ready to go. We can get the latest app version by getting the first item in the array returned by this endpoint: https://api.appcenter.ms/v0.1/apps/${teamName}/${appName}/distribution_groups/${distGroup}/releases. Calling the endpoint with axios looks like this:
export async function fetchLatestRelease(teamName, appName, distGroup) {
try {
const response = await axios.get(`https://api.appcenter.ms/v0.1/apps/${teamName}/${appName}/distribution_groups/${distGroup}/releases`,
{
headers: {
'X-API-Token': settings.appCenterApiKey
}
});
return response.data[0];
} catch (error) {
console.error(error);
return '';
}
}
It should return:
{
id: 1337,
short_version: '1.0.0',
version: '1337',
origin: 'appcenter',
uploaded_at: '2020-03-02T12:15:05.146Z',
mandatory_update: false,
enabled: true,
is_external_build: false
}
We only need to pipe the id on to https://api.appcenter.ms/v0.1/apps/${teamName}/${appName}/releases/1337 to get the details of this release. Keep in mind that I redacted some of these values.
export async function fetchAppCenterRelease(teamName, appName, releaseId) {
try {
const response = await axios.get(`https://api.appcenter.ms/v0.1/apps/${teamName}/${appName}/releases/${releaseId}`,
{
headers: {
'X-API-Token': settings.appCenterApiKey
}
});
return response.data;
} catch (error) {
console.error(error);
return '';
}
};
It should return:
{
app_name: 'Dummy-app',
app_display_name: 'Dummy app',
app_os: 'iOS',
is_external_build: false,
origin: 'appcenter',
id: 1337,
version: '1337',
short_version: '2.0.0',
size: 1337,
min_os: '9.1',
device_family: 'iPhone/iPod',
bundle_identifier: 'demo.staging',
fingerprint: '[REDACTED]',
uploaded_at: '2020-03-02T12:15:05.146Z',
download_url: '[REDACTED]',
install_url: '[REDACTED]',
enabled: true,
provisioning_profile_type: 'adhoc',
provisioning_profile_expiry_date: '2020-09-12T10:11:13.000Z',
provisioning_profile_name: 'Adhoc dummy app',
is_provisioning_profile_syncing: false,
release_notes: 'Did some app fixes.',
package_hashes: [
'[REDACTED]',
'[REDACTED]'
],
destinations: [
{
destination_type: 'group',
id: '00000000-0000-0000-0000-000000000',
name: 'dummy'
}
],
build: {
commit_hash: '0000000000000000000000000000000',
commit_message: 'Fix spelling in actions'
},
destination_type: 'group',
distribution_groups: [
{
id: '00000000-0000-0000-0000-000000000',
name: 'dummy'
}
]
}
You get a lot of data here. download_url is the property we're looking for. Then we proceed to upload the app to the App Automate suite.
export async function uploadApp(url) {
try {
const body = new FormData();
body.append('data', `{\"url\": \"${url}\"}`);
const { data } = await axios.post(
'https://api-cloud.browserstack.com/app-automate/upload',
body,
{
headers: {
'Content-Type': `multipart/form-data; boundary=${body.getBoundary()}`
},
auth: {
username: settings.browserstackUsername,
password: settings.browserstackSecret
}
});
return data;
} catch (error) {
console.error(error);
return null;
}
}
When successful, Browserstack will return { app_url: 'bs://[some-hash]' }. This url can be inserted into your web driver and we're ready to create some tests!
Simple test suite with Web Driver and App Automate
You might be familiar with Selenium, but have you heard about Appium? They're pretty similar, however Appium is targeted towards native mobile apps rather than the browser. Now we can configure our tests. This is where we can choose OS, phone make, etc. wd makes this job easy. The web driver client works as a test framework. This allows you to interact with the mobile app. Just define up the configuration for an iPhone X.
let capabilities = {
'device': 'iPhone X',
'os_version': '11',
'name': 'Bstack Sample Test',
'browserstack.user': settings.browserstackUsername,
'browserstack.key': settings.browserstackSecret,
'build': 'iOS',
'name: 'test',
'app': app_url,
'browserstack.debug': true,
'browserstack.networkProfile': '4g-lte-high-latency'
}
Here you can define device and OS version. Some properties are obviously used as settings for browserstack. Full documentation on capabilities can be found here.
After defining the capabilities, we can initialize the web driver and make our test!
let capabilities = {
'device': 'iPhone X',
'os_version': '11',
'name': 'Bstack Sample Test',
'browserstack.user': settings.browserstackUsername,
'browserstack.key': settings.browserstackSecret,
'build': 'iOS',
'name: 'test',
'app': app_url,
'browserstack.debug': true,
'browserstack.networkProfile': '4g-lte-high-latency'
}
const driver = wd.promiseRemote("http://hub-cloud.browserstack.com/wd/hub");
try {
await driver.init(capabilities);
const emailfield = await driver.waitForElementById('emailLoginTextField', asserters.isDisplayed, 10000, 100);
await emailfield.sendKeys("example@gmail.com");
const passwordfield = await driver.waitForElementById('passwordLoginTextField', asserters.isDisplayed, 10000, 100);
await passwordfield.sendKeys("test");
const loginButton = await driver.waitForElementById('confirmLoginButton', asserters.isDisplayed, 10000, 100);
await loginButton.click();
assert(true);
} catch (error) {
console.error(error);
assert(false);
}
driver.quit();
This test logs in a user for iPhone X version 11. It shows off tapping and typing. There are many possibilities in wd for interacting with the mobile device. The tooling in BrowserStack shows off a lot of cool features like video playback and action-to-action screenshots. I wrote a script that handles the running of the tests across multiple devices and versions. This script runs in sequence and is triggered by a cron job once every weekday (0 8 * * 1-5) in our team.
Thank you for reading my post! I hope to explore more topics on Appium in the future and might even write about app upgrade testing one day.