VMware GSX Server 2.5.2

Unfortunately, keyboard support for the PC (virtual or otherwise) is a complex affair. To do it justice, we have to start with some background information — greatly simplified.

Pressing a key on the PC keyboard generates a scan code based roughly on the position of the key. For example, the Z key on a German keyboard generates the same code as the Y key on an English keyboard, because they are in the same position on the keyboard. Most keys have one-byte scan codes, some keys have two-byte scan codes with prefix 0xe0.

Internally, GSX Server uses a simplified version of the PC scan code that is a single 9-bit numeric value, called a v-scan code. A v-scan code is written as a three-digit hexadecimal number. The first digit is 0 or 1. For example, the left-hand Ctrl key has a one-byte scan code (0x1d); its v-scan code is 0x01d. The right-hand Ctrl key scan code is two bytes (0xe0, 0x1d); its v-scan code is 0x11d.

An X server uses a two-level encoding of keys. An X key code is a one-byte value. The assignment of key codes to keys depends on the X server implementation and the physical keyboard. As a result, an X application normally cannot use key codes directly. Instead, the key codes are mapped into keysyms that have names like space, escape, x and 2. The mapping can be controlled by an X application via the function XChangeKeyboardMapping() or by the program xmodmap. To explore keyboard mappings, you can use xev, which shows the key codes and keysyms for keys typed into its window.

To recap, a key code corresponds roughly to a physical key, while a keysym corresponds to the symbol on the key top. For example, with an XFree86 server running on a PC, the Z key on the German keyboard has the same key code as the Y key on an English keyboard. The German Z keysym, however, is the same as the English Z keysym, and different from the English Y keysym.

For an XFree86 server on a PC, there is a one-to-one mapping from X key codes to PC scan codes (or v-scan codes, which is what GSX Server really uses). GSX Server takes advantage of this fact. When it is using an XFree86 server on the local host, it uses the built-in mapping from X key codes to v-scan codes. This mapping is keyboard independent and should be correct for most, if not all, languages. In other cases (not an XFree86 server or not a local server), GSX Server must map keysyms to v-scan codes, using a set of keyboard-specific tables.

Key code mapping is simple, automatic and foolproof. (Keysym mapping is more complex and described later.) However, because the program cannot tell whether a remote server is running on a PC or on some other kind of computer, it errs on the safe side and uses key code mapping only with local X servers. This is often too conservative and has undesirable effects. Luckily, this and other behavior related to key code-mapping can be controlled by using a text editor to add configuration settings to the virtual machine's configuration file.

• xkeymap.usekeycodeMapIfXFree86 = true
  Use key code mapping if using an XFree86 server, even if it is remote.

• xkeymap.usekeycodeMap = true
  Always use key code mapping regardless of server type.

• xkeymap.nokeycodeMap = true
  Never use key code mapping.

• xkeymap.keycode.<code> = <v-scan code>
  If using key code mapping, map key code <code> to <v-scan code>. In this example, <code> must be a decimal number and <v-scan code> should be a C-syntax hexadecimal number (for example, 0x001).

The easiest way to find the X key code for a key is to run xev or xmodmap -pk. Most of the v-scan codes are covered in the V-Scan Code Table. The keysym mapping tables described below are also helpful.

Use this feature to make small modifications to the mapping. For example, to swap left control and caps lock, use the following lines:

xkeymap.key code.64 = 0x01d # X Caps_Lock -> VM left ctrl
xkeymap.key code.37 = 0x03a # X Control_L -> VM caps lock

These configuration lines can be added to the individual virtual machine configuration, to your personal GSX Server configuration (~/.vmware/config), or even to the host-wide (/etc/vmware/config) or installation-wide (usually /usr/local/lib/vmware/config) configuration.

When key code mapping cannot be used (or is disabled), GSX Server maps keysyms to v-scan codes. This is done using one of the tables in the xkeymap directory in the GSX Server installation (usually /usr/local/lib/vmware).

Which table you should use depends on the keyboard layout. The normal distribution includes tables for PC keyboards for the United States and a number of European countries and languages. And for most of these, there are both the 101-key (or 102-key) and the 104-key (or 105-key) variants.

GSX Server automatically determines which table to use by examining the current X keymap. However, its heuristics may sometimes fail. In addition, each mapping is fixed and may not be completely right for any given keyboard and X key code-to-keysym mapping. For example, a user may have swapped control and caps lock using xmodmap. This means the keys are swapped in the virtual machine when using a remote server (keysym mapping) but unswapped when using a local server (key code mapping).

Therefore, keysym mapping is necessarily imperfect. To make up for this defect, most of the behavior can be changed with configuration settings:

• xkeymap.language = <keyboard-type>
  Use this if GSX Server has a table in xkeymap for your keyboard but can't detect it. <keyboard-type> must be one of the tables in the xkeymap directory. (See above for location.) However, the failure to detect the keyboard probably means the table isn't completely correct for you.

• xkeymap.keysym.<sym> = <v-scan code>
  If you use keysym mapping, map keysym <sym> to <v-scan code>. When you do, <sym> must be an X keysym name and <v-scan code> should be a C-syntax hexadecimal number (for example, 0x001).

  The easiest way to find the keysym name for a key is to run xev or xmodmap -pk.

  The X header file /usr/X11R6/include/X11/keysymdef.h has a complete list of keysyms. (The name of a keysym is the same as its C constant without the XK_ prefix.) Most v-scan codes are in V-Scan Code Table.

  The xkeymap tables themselves are also helpful. Use them to fix small errors in an existing mapping.

• xkeymap.fileName = <file-path>
  Use the keysym mapping table in <file-path>. A table is a sequence of configuration lines of the form
  <sym> = <v-scan code>
  where <sym> is an X keysym name, and <v-scan code> is a C-syntax hexadecimal number (for example, 0x001). (See the explanation of xkeymap.keysym above for tips on finding the keysyms and v-scan codes for your keyboard.)

  Compiling a complete keysym mapping is hard. It is best to start with an existing table and make small changes.