Remote AutoTVM with Windows

During my last story, I wrote that AutoTVM does not work locally. Based on this notion, the apparent next step is to try to run it remotely, I will demonstrate this using a Raspberry Pi. I feel that current resources are a little bit scattered and it is difficult to start doing something with it.

A summary of the entire workflow:

1. Install LLVM on Windows. Make sure you build LLVM with your target machine’s architecture.
2. Install TVM and its Python package on Windows.
3. Install TVM runtime on Raspberry.
4. Start tracker from TVM, that manages server on some computer.
5. Start server on Raspberry Pi to register server to tracker.
6. Run TVM script on Windows.

To begin with, we start with building LLVM on Windows. Now, we specifically want to compiler for other architectures.

regsvr32 "%VSINSTALLDIR%\DIA SDK\bin\msdia140.dll"
regsvr32 "%VSINSTALLDIR%\DIA SDK\bin\amd64\msdia140.dll"

pip install psutil
git clone https://github.com/llvm/llvm-project.git llvm

cmake -S llvm\llvm -B build -DLLVM_ENABLE_PROJECTS=clang -DLLVM_TARGETS_TO_BUILD=X86;AArch64,ARM -Thost=x64 -DCMAKE_INSTALL_PREFIX=c:\libraries\llvm
exit

Adjust DLLVM_TARGETS_TO_BUILD according to your needs. In my case, I am using a Raspberry Pi 4b with a 64-bit OS, so AArch64 will fulfill my needs.

Next, build the created solution in the build folder on release mode. Add the installed directory to the environmental variable.

Build TVM on Windows

I will assume you have cmake and Python ready. Personally, I like to use cmake-gui for some extra help.

git clone --recursive https://github.com/apache/tvm tvm
cd tvm
mkdir build
cmake --build build --config Release -- /m

Then go to the build directory and use cmake to change some settings.

Set “CMAKE_INSTALL_PREFIX” in the directory where you want to install it, maybe next to llvm.

Since we have put llvm in environment path we can set

USE_LLVM llvm-config — link-static

You can also set openMP

Then again build with Visual Studio.

Go to the root of TVM and run

pip install -e /tvm/python

To install the Python module.

You should update your numpy, otherwise you will get a page error.

Build Raspberry Pi TVM Runtime

Now get your raspberry pi and build TVM runtime.

https://tvm.apache.org/docs/how_to/deploy_models/deploy_model_on_rasp.html

git clone --recursive https://github.com/apache/tvm tvm
cd tvm
mkdir build
cp cmake/config.cmake build
cd build
cmake ..
make runtime -j4
export PYTHONPATH=$PYTHONPATH:~/tvm/python

Finally, we are ready to experiment.

Start tracker

The tracker manages autoTVM server clusters, you can run this on your Windows computer with.

python -m tvm.exec.rpc_tracker --host=192.168.0.59 --port=9190

Change the host to your Windows computer IP.

Start Server

On the server side make sure the python folder is exported

export PYTHONPATH=$PYTHONPATH:~/tvm/python

Then run

 python -m tvm.exec.rpc_server --tracker=192.168.0.59:9190 --key=rpi

The tracker IP is the IP of the computer you started your tracker with.

Use the same key for devices of the same type.

To check whether you have succeeded, you can use

python -m tvm.exec.query_rpc_tracker --host=192.168.0.59 --port=9190

Again, change the host IP to the tracker IP. You should see something like this

notice the rpi server exists

Run AutoTVM

https://drive.google.com/file/d/1zb8w0m2_eW-Yn-5rZ-ivp5dpoKSG_YlP/view?usp=drive_link

Now you can run this script, which is modified from here with reduced workload. Change the trackerIP and port inside the script.

You will see something like this.

Feel free to message me if you have any problems with this story!