Re-Training test-ai-classifier using Google Colab
An AI classifier called test-ai-classifier (detalis in Reference section) for Appium is open sourced. This post will show you how to build the classifier using Google Colab, customize them for your application, and add it to Appium.
Steps to re-train:
Step1: Create zip file
git clone https://github.com/testdotai/classifier-builder.git
Create Zip file — training_images.zip
Step2: Upload to google drive (Example: digitransolutions5@gmail.com)
Upload training_images.zip to google drive and share.
Step3: Create a sharable link to anyone
https://drive.google.com/file/d/1q9ZcSU7JtjcZPbXLtrsbpyUgsGPwj2er/view?usp=sharing
So id = 1q9ZcSU7JtjcZPbXLtrsbpyUgsGPwj2er
Step4: Copy zip file to colab
!wget — load-cookies /tmp/cookies.txt “https://docs.google.com/uc?export=download&confirm=$(wget — quiet — save-cookies /tmp/cookies.txt — keep-session-cookies — no-check-certificate ‘https://docs.google.com/uc?export=download&id=1q9ZcSU7JtjcZPbXLtrsbpyUgsGPwj2er' -O- | sed -rn ‘s/.*confirm=([0–9A-Za-z_]+).*/\1\n/p’)&id=1q9ZcSU7JtjcZPbXLtrsbpyUgsGPwj2er” -O training_images.zip && rm -rf /tmp/cookies.txt
Step5: Unzip training_images.zip
!unzip -q training_images.zip
Step6: Create colab project
Take re-train-classifier.ipynb from
https://github.com/anbunathan/appium-examples/tree/master/classifier-builder
Upload re-train-classifier.ipynb to colab
Create tmp and output folders
Run re-train-classifier.ipynb
Step7: Zip tmp and output folders
!zip -r tmp.zip tmp
!zip -r output.zip output
Download the file:
from google.colab download the files
files.download(“tmp.zip”)
files.download(“output.zip”)
Step8: Copy this zip file into Google drive
from google.colab import drive
drive.mount(‘/content/gdrive’)
!cp tmp.zip /content/gdrive/My\ Drive/
!cp output.zip /content/gdrive/My\ Drive/
Step9: Convert output/saved_model.pb to tensorflowjs
In colab,
!pip install tensorflow==1.13.1 tensorflow_hub==0.5.0 tensorflowjs==0.8.6
!tensorflowjs_converter — input_format=tf_frozen_model — output_node_names=final_result output/saved_model.pb web_model
This creates a web_model folder.
Step10: Copy the contents of web_model folder
Copy the contents of this folder to the npm dependency install for ‘appium-classifier-plugin’ under the model folder
!zip -r web_model.zip web_model
download zip file to local PC
copy web_model contents from
D:\trial\appium_examples\classifier-builder\colab\web_model 25dec20\web_model
to
C:\Users\natha\AppData\Roaming\npm\node_modules\test-ai-classifier\model\
after renaming tensorflow_model.pb to model.pb, model.json to weights.json. group1-shard1of1 as it is.
Step10: Pre-requisites
Server installation:
cd C:\Users\natha\AppData\Roaming\npm\node_modules/appium/node_modules/appium-uiautomator2-server/apks/
adb install appium-uiautomator2-server-v4.15.0.apk
adb install appium-uiautomator2-server-debug-androidTest.apk
Capabilities:
capabilities.setCapability(“automationName”, “UiAutomator2”);
capabilities.setCapability(“skipServerInstallation”, “true”);
Step11: Sample run
https://github.com/testdotai/classifier-builder
add saved_model.pb and saved_model.pbtxt under
D:\trial\appium_examples\classifier-builder\sample_run\testai_model
from pycharm run -> run_model.py under testai_model (classifier-builder project)
or
from commandline execute,
C:\python36\python.exe run_model.py
//python run_model.py — image cart.png
Observe cart probability = 0.685084
Note: the following change in run_model.py is required:
parser.add_argument(“ — image”, help=”image to be processed”)
to
parser.add_argument(
‘ — image’,
type=str,
nargs=’?’,
default=’cart.png’,
help=’image to be processed.’
)
Step12: Test the plugin
Go to your command line and start appium(appium -a 127.0.0.1 -p 4723).
Open IntelliJ:
Take TestAppiumAI.java from
https://github.com/anbunathan/appium-examples/tree/master/Template1/src/test/java/com/example/appium
Run testngAICLI.xml (it invokes TestAppiumAI.java)
Also, I have added a video: https://youtu.be/CFDSejLJEQY
— — — — — — — — — — — — — — — — — — — — — — — — — — — —
Reference:
Training Data for App Classifier
https://medium.com/testdotai/training-data-for-app-classifier-f217dc005523
test.ai repo — Classify Elements
https://github.com/testdotai/classifier-builder
https://github.com/testdotai/classifier-builder.git
Test.ai’s Appium plugin: How does it work, and how accurate is it?
https://david.rothlis.net/testdotai-appium-plugin/
https://github.com/drothlis/classifier-builder
Dealing with files in colab
https://stackoverflow.com/questions/50453428/how-do-i-download-multiple-files-or-an-entire-folder-from-google-colab
https://neptune.ai/blog/google-colab-dealing-with-files
https://datascience.stackexchange.com/questions/64131/how-to-save-dynamic-files-from-colab-to-google-drive
Convert to tensorflowjs files:
https://github.com/tensorflow/tfjs-converter/tree/master/tfjs-converter
Downloading Datasets into Google Drive via Google Colab
https://towardsdatascience.com/downloading-datasets-into-google-drive-via-google-colab-bcb1b30b0166
Adding AI to Appium Step-by-Step guide for Android emulator
https://www.linkedin.com/pulse/adding-ai-appium-step-by-step-guide-android-emulator-alejandro-avella/
Sample test.ai Appium AI plugin
https://www.youtube.com/watch?v=CzREoMUQxZo
Common Mobile/Web App Icons
A collection of scraped icon images from the web and mobile apps
https://www.kaggle.com/testdotai/common-mobile-web-app-icons
argparser default value
https://stackoverflow.com/questions/15301147/python-argparse-default-value-or-specified-value
— — — — — — — — — — — — — — — — — — — — —
Contact:
Mail: anbunathan.r@gmail.com
Website: http://digitranssolution.com/
