Linux on Windows — Take Two (Wsl 2)

Intro

A long time ago there was a flame war between Windows OS and Linux OS. It was a fight against the evil empire of the monopolistic desktop OS vs. the fanatics of open source. It’s important to mention this because of the reluctance to accept the new Microsoft (which is still here) among hard core Linux fans.

Now we live in different times, as  Microsoft embraces open source in their own products (DotNet framework for instance), and produces software for multiple platforms, including the famous free and great Microsoft Visual Studio Code.

Dual nature of a developer’s life

In the world of enterprise software for many years we used to live in a dichotomy:  the desktop OS is Windows and the server OS is Linux.

For many years Windows desktop developers have been looking for an easier way to work within the Linux environment. This includes thousands of our Avenga developers working on ambitious digital products.

Linux based laptops

Yes, there are Linux based laptops which are designed and fully supported while running  Linux, but they are quite rare (probably Dell is the only well known exception).

Plus you cannot install Windows specific software on them if you need to without involving virtual machines; like Microsoft Office or specific tools for hardware.

Just install the Linux solution

What is easier than just taking the pen drive and starting the Linux OS installer. Everything seems to run smoothly unless it doesn’t. It’s really hard to make all the hardware work properly or with the same efficiency as in Windows OS, for which this hardware was built. It simply wasn’t tested on Linux and wasn’t designed for Linux.

It’s up to the Linux community to catch up with the latest hardware and firmware of devices. Usually, the support (truly working hardware) comes after a few months or years . . . or never.

The Linux community focuses mainly on server hardware and does not have the resources to keep up with the extreme diversity of desktop hardware.

And the hardware producers themselves often ‘forget’ to add Linux installers in their motherboards, cameras, speakers, and other devices. Or even when they do add them, the quality and number of features are a mere subset of what is available to Windows 10 users.

So the problem is with the very word “just” – it usually means tens of hours wasted on playing with different kernel modules, compiling alternative kernels, modprobing, removing, and reinstalling entire OS etc. or accepting missing features and lower performance. We’ve all been there.

The macOS solution

Many chose Macbooks as their development machines for the proximity of UNIX, based in macOS and Linux. They are not the same, there are different operating systems, and are not compatible with each other.

But the bash/zsh terminal has many similar commands. It feels very much UNIX-like, much closer to Linux than the Windows command line interface.

Slashes are slashes, etc. The difference between the two environments was acceptable, plus the high quality and longevity of Apple hardware made it a default choice for many developers.

Windows’ command line tools

CygWin and Git Bash and others represent different approaches; a subset of Linux command line tools and some emulation for a filesystem to help it pretend to be Linux-like. There’s grep, awk, etc. so it’s closer to Linux than bare Windows, definitely. But, once you want to run anything outside the prepackaged toolset,  you are immediately reminded, it’s not even close to Linux.

SSH to remote machine

This was kind of the ultimate solution for SSH to the actual Linux machine and to work on a true native OS. The problem? You need another machine and working with the local files is cumbersome (ok, there’s scp and other tools, but the experience is definitely not seamless).

Virtual Machine (VMWare, HyperV, Virtual Box)

Another approach is to run a virtual machine locally. The problem however, is that it takes a lot of memory, the performance in the interactive mode (actually using the command line) starts slowly, and the integration is disappointingly slow and frustrating. Plus, in order to make it less painful, you needed to install a modified kernel with integration modules which involves kernel compilations, headers, etc. and  this is really unproductive.

Microsoft loves Linux (take two)

Take one

The Windows Subsystem for Linux v1 (WSL1) was an emulation of the Linux subsystem, through translating commands to Windows kernel. It worked with an initial low overhead but also had low performance and significant compatibility problems.

So let’s focus on WSL2 (Windows Subsystem for Linux v2) which is based on a lightweight virtual machine and well integrated with a host OS.

Just run it

Installation requires the latest (2004) version of Windows. So upgrade your Windows OS first.

Then it’s a question of copying and pasting a few command line commands.

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

wsl --set-default-version 2

Then we go to the Microsoft Store and . . . install our Linux distribution of choice.

https://www.microsoft.com/en-gb/p/ubuntu-2004-lts/9n6svws3rx71?activetab=pivot:overviewtab

(Linux in the Microsoft Store? Has hell already frozen over?).And the new Terminal supports WSL2 perfectly.

https://www.microsoft.com/en-gb/p/windows-terminal/9n0dx20hk701?activetab=pivot:overviewtabAnd it works!

Installing dev tools

Git is already installed.

Nodejs is installed with:

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -

sudo apt install nodejs

Let me use the example from our VR demo which is available on GitHub.

Let’s build it and run it.npm install three

npm install webpack

npx webpack

Visual Studio Code surprise

I was really surprised, in a very positive way, when I tried it for the first time.It installs VS Code Server and launches VS Code GUI on Windows.

And the VS Code has now access to our Linux folder! It is still benefiting from the optimized Windows GUI.

Docker

It’s hard to imagine modern software development without Docker containers.

Yes, there’s a client for Windows that has been around for ages, but what does it look like now, with WSL2?

It looks very good. Docker for Windows is prepared for WSL2 and actually has built-in support for this new option.So Docker supports  WSL2 already!

Machine learning with GPU acceleration

Unfortunately in the current (June 2020) production release of Windows 10 and WSL2, there’s no access to a GPU.

But access is coming in the following months; and it’s not just a promise but it already is working in the preview versions of Windows 10 and WSL2.

Originally I planned to cover this part in the next version of this article, but let’s try with a Fast Ring Windows experimental build and experimental NVIDIA CUDA for WSL.

Let’s start with updating WSL.

NVIDIA CUDA for WSL is an exclusive download for NVIDIA Developers, so I had to join the program first and then download and install the special driver.

Now let’s try to use Nvidia GPU containers from WSL2.

We have to enable the experimental repositories first.And finally we install:

sudo apt-get install -y nvidia-docker2

Unfortunately, it didn’t work . . .

I had to remove the docker:

sudo apt remove docker

And then install the docker by:

curl https://get.docker.com | sh

And start it manually with:

sudo dockerd

Let’s try the benchmark now with:

docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmarkAnd yes! It’s using the GPU acceleration, and the NVIDIA video card is listed as a device used for a machine learning benchmark.

So CUDA under WSL2 is working already in the preview releases!

A quick comparison against the native Linux (GPU ML container)

Let’s run it on the docker under native Ubuntu. So, first reboot. And after launching the command from the Ubuntu machine, we see the results are similar to the code executed on the GPU directly and it uses GPU memory only.

The actual performance numbers vary significantly from test to test, and in both cases they are in the similar range, so the performance is at least comparable.

Missing parts

Graphical UI applications

The command line is the primary way of interacting with the Linux subsystem, however there are times when the GUI app has to be launched. Support for these types of applications is promised in the future, however now the workaround is to install xserver and connect to it; but unfortunately there is no hardware acceleration available. It should be available in the following months (Microsoft demonstrated it working on the conference live).

Not a big deal for many, as we can run a fully optimized browser on our Windows with full hardware acceleration and desktop integration.

All the major players, NVIDIA, AMD, and Intel have released their preview version of drivers for WSL2 to enable hardware acceleration; so it’s happening now.

Snap store

Ubuntu and other distributions promote alternative ways of installing apps such as Snap. Unfortunately it’s not working (yet?).

Init subsystem

There is no init – commands like reboot, because everything that depends on the init subsystem is not working.

CentOS / Fedora

In the Microsoft Store there is Ubuntu 18.04 LTS and 20.04 LTS, SUSE, Debian, Alpine, and Kali, but from the RedHat/CentOS/Fedora family I can only find a paid version of Fedora Remix which is a custom fork of Fedora.

Summary

Windows, no matter if you hate it or love it or are somewhere in between, opens up the entire universe of diverse hardware, with different performance statistics, and different price ranges.

Yes, now you can have it all – the perfect native support for the hardware with tools and runtimes of near native Linux. And at a very low cost in terms of time spent to install it and the low performance overhead later.

Linux fanatics seeing Ubuntu promote WSL2 are outraged. There are comments about betrayal, saying it’s another trick from Microsoft to hurt Linux; just another sneaky attempt in the now cold war against the open source community.

Now it is Windows users who can have more Linux-like workstations, than MacOS users. It all happened in the first half of 2020 and IT history will never forget his moment. What an interesting time!

Thank you for your new attitude, both Microsoft and Ubuntu, and others who made it happen!