Arduino Project: Platform Independent Clipboard

Posted by & filed under How I solved my problem, In English, Tech.

The Platform Independent Clipboard

The source code for the project is available on the GitHub: https://github.com/bitblueprint/Arduino-UniversalClipboard – this article references line numbers in the 596944 commit. This source code is released under an open MIT License, allowing you to copy, modify and redistribute it with your changes, commercially or not. If you like the project – please ping me on Twitter, I love when people get something out of whatever I publish.

Who would need such a thing?

The idea of a platform independent clipboard first came to me when a relative working at the state national police force of Denmark, introduced me to a challenge of theirs.

When a police officer is dealing with a case, it involves repeatedly entering name and birthdates into several computer systems. Some of these systems are different applications on the same computer, some are on different physical computers and some of these have entirely different operating systems. This part of the officer’s daily work has been identified by consultants as time consuming and error prone – resulting in additional time wasted, when correcting data or not finding the correct persons, due to mistakes made when typing.

Word of caution – this is almost password sniffing key-logger

With minor changes of the device’s source code, it could act as a key-logging device, that could potentially snoop/sniff/eavesdrop on classified information if hidden from the person using the computer. It is essentially a device performing a man in the middle attack.

Therefore it is important to permanently force the activation pin low, once this device is introduced in into a real police station. This way tinkers have a much harder time connecting to the Arduino in a state where it is reprogrammable. Alternatively change the source code to not wait for an activation signal before acting as a keyboard towards the computer.

Introduction

This project aimed at designing and implementing a product prototype, utilizing both hardware and software, to propose a viable solution for the aforementioned problem.

The product should help the police officer transfer confidential data such as names and birthdates from one physical machine to another, reducing the time spent and the errors occurring when retyping the same data into multiple computer systems.

Requirements

From an initial conversation with two officers, I formulated an ordered list of functional requirements – an initial backlog of user stories:

  1. As an officer, I want to be able to use the keyboard as usual, so that I can perform my daily work without any additional challenges introduced by technology.
  2. As an officer, I want to selectively start recording a sequence of keystrokes starting from the press of “a special key” until the press of the another or the same special key so that I can control from when to start the recording of my keystrokes.
  3. As an officer, I want to be able to replay a sequence of keystrokes previously recorded, so that I can retype names, numbers, etc. without introducing typing errors.
  4. As an officer, I want to be able to record and playback multiple sequences keystrokes so that I can store and replay more than one sequence of keystrokes.
  5. As an officer, I want to be able to transfer the recorded keystrokes between physical computers on a medium so that I can reuse the a recorded sequence from one machine on another machine.
  6. As an officer, I want to be able to encrypt and password protect my transfer medium (NFC chip) so that I do not have to worry that confidential data is compromised if I loose the transfer medium.

As qualitative / non-functional requirements, the following observations are made

  1. The cost of the device matters – we want a solution that is not too expensive in production. 80 euro (~ 600 DKK) per device seems to be a reasonable upper limit.
  2. Perhaps needless to say, the device has to be user friendly. Meaning that an officer should be able to operate the universal clipboard with little to no training and with no more instructions that what can be written on a sticky label on-top of the device.

What hardware is needed?

Why not just implement this in software? Well, as the problem suggests, this may take a long time as we are talking about a couple of different applications on a couple of different operating systems. Figuring out the plumbing, how to write a slick user interface across the broad range of configurations would probably end up taking too much time.

So I thought to myself – could this be implemented independently of the software on the PC? This was what I wanted to do.

These is the complete list of parts that you’ll need to build one platform independent clipboard:

This totals 563 DKK per device or alternatively 70 euro (~521 DKK) when producing 100 or more in a batch.

Implementing user story #1 – Being able to use the keyboard as usual

So – a user want to connect a device to her computer and her keyboard to the device, after which she expects the computer to respond to key presses made on the keyboard, as if nothing was in the middle.

Hardware compatibility issues encountered early on in the project

Initially I tried implementing user story #1 using the Arduino Due – I figured it had two USB ports, one that could act as a native USB device towards the computer and one that act as USB host towards the keyboard.

Unfortunately it turned out that the programming USB port on the Due, can only easily be used for programming the device. The native USB port can be used for either acting as a client (keyboard, mice, etc) towards the computer or as a host for peripherals, but of course not both at the same time.

The article about the Due on Arduino.cc, talks about the native USB port:

Using the Native port enables you to use the Due as a client USB peripheral (acting as a mouse or a keyboard connected to the computer) or as a USB host device so that devices can be connected to the Due (like a mouse, keyboard, or an Android phone).

Bummer – I tried extending the Due with a USB host shield, but it seemed like the Due got confused and thought I was asking the native port to act as a USB host device, even when using the libraries provided with the host shield.

I then tried implementing user story #1 using the Arduino Uno and the USB host shield – this should in principle be possible, but the Uno board’s firmware has to be flashed between each reprogramming of the board, as it needs different firmware for the board to act as a keyboard towards the computer than the firmware needed to act as an Arduino to the computer. Read more about flashing the Uno to act as a peripheral to the computer on this hackaday.com article.

What to do? Go search the internet for more information.

It turns out that the Arduino Leonardo board has an interesting feature very suitable for this application. Its single USB port can act as both a programming port, enabling serial communication towards the computer while at the same time acting as several peripherals towards the PC.

Combined with a USB host shield the Arduino Leonardo gives the unique possibility of being able to listen for ingoing keystrokes from a keyboard connected to the USB host shield while being able to send these keystrokes onto the PC through the USB port on the Leonardo.

Registering key strokes from the USB host shield

This is done using the USB host shield, through the library provided by Oleg Mazurov and Alexei Glushchenko from circuits@home, Kristian Lauszus and Andrew Kroll on GitHub (download).

Once the library has been installed, the projects source code line #143 and #146 initiates global objects USBHost and HidKeyboard that will enable communication with a keyboard connected to the USB Host shield.

USB UsbHost;
HIDBoot HidKeyboard(&UsbHost);

Once initialized, the HidKeyboard object has a HIDReportParser named keyboard_in (initialized on line #157) registered on line #339 and the USB host shield is given program control on every loop-call that the Arduino does – see line #362

UsbHost.Task();

The ProxyKeyboardParser class, which is implemented for this particular project and extended from the KeyboardReportParser parser (which is again a specialization of the HIDReportParser), provides callback methods for the HidKeyboard object to call when the user interacts with the keyboard.

Sending key strokes onward to the connected computer

This step was the most trouble I had in the project. The Arduino keyboard library functionality kindly translates ASCII characters to a KeyReport object with key strokes that the computer understands. This way it is easy to send upper and lower-case letters using the Keyboard.write method. But if you have the raw key strokes and the modifier byte (communicating the left/right ctrl, shrift, alt and GUI key states), this turns out to be an anti-feature.

I looked into the Arduino cores implementation of the press and release methods and found that I could extend the Keyboard_ class and overwrite the implementation (lines 88-123 in the projects source code) to not do a transformation from ASCII to key stroke bytes and change of modifiers.

// Sending out a key press to the computer
size_t KeyboardOut::press(uint8_t k) {
  uint8_t i;
  // Add k to the key report only if it's not already present
  // and if there is an empty slot.
  if (_keyReport.keys[0] != k && _keyReport.keys[1] != k && 
    _keyReport.keys[2] != k && _keyReport.keys[3] != k &&
    _keyReport.keys[4] != k && _keyReport.keys[5] != k) {
 
    for (i=0; i<6; i++) {
      if (_keyReport.keys[i] == 0x00) {
        _keyReport.keys[i] = k;
        break;
      }
    }
    if (i == 6) {
      setWriteError();
      return 0;
    }	
  }
  send_report();
  return 1;
}
 
// Sending out a key release to the computer
size_t KeyboardOut::release(uint8_t k) {
  uint8_t i;
  // Test the key report to see if k is present.  Clear it if it exists.
  // Check all positions in case the key is present more than once (which it shouldn't be)
  for (i=0; i<6; i++) {
    if (0 != k && _keyReport.keys[i] == k) {
      _keyReport.keys[i] = 0x00;
    }
  }
  send_report();
  return 1;
}

This functionality is called from the ProxyKeyboardParser class (lines 159-180). A simplified version of the source code below, shows how key presses from the USB host shield is communicated to the computer using the keyboard_out object.

void ProxyKeyboardParser::OnControlKeysChanged(uint8_t before, uint8_t after) {
  keyboard_out.set_modifiers(after);
  keyboard_out.send_report();
  working_blink();
}
 
void ProxyKeyboardParser::OnKeyDown(uint8_t mod, uint8_t key)
{
  keyboard_out.press(key);
  working_blink();
}
 
void ProxyKeyboardParser::OnKeyUp(uint8_t mod, uint8_t key)
{
  keyboard_out.release(key);
  working_blink();
}

The working LED blinks whenever a key press or release is intercepted – to show the user that something is happening, in the hopes of making the device more user friendly.

Implementing user story #2 – Being able to record keystrokes

In order for the user to be able to record their keystrokes, she must first activate the recording. This is done by pressing a special combination of keys. I thought of a sequence which I anticipated was rarely used when entering data into the systems at the police force. This has remains to be finally verified, but the code can easily be modified to support an alternative activation key sequence.

The ProxyKeyboardParser’s OnKeyDown method was extended with a call to an axillary function which evaluates if the user is trying to start or stop a recording.

void ProxyKeyboardParser::OnKeyDown(uint8_t mod, uint8_t key)
{
  bool controlled = intercept_recording_command(mod, key);
  if(!controlled) {
    intercept_recording_key(mod, key);
    keyboard_out.press(key);
  }
  working_blink();
}

The implementation of the intercept_recording_command function checks for three scenarios:

  1. Start the recording of keystrokes. (line 234)
  2. Replay a recording of keystrokes. (line 238)
  3. Stop the recording of keystrokes. (line 242)

The replaying of keystrokes are for the next user story to implement, whereas this focusses on starting and stopping a recording.

When the user holds down the left or right control key, while holding down the left or right shift key – and presses one of the numbers one though nine on the numeric pad, the recording of a sequence of keystrokes begins. See lines 234-237.

When recording is active and the user presses either the tabular key, the return/enter on either the main keyboard or the numeric pad, the recording is stopped.

When starting  are recording the an integer variable active_channel_index is set to zero and a call to the clear_channel function clears out the recording buffer (an array called channels), by writing zeros all in its entire length.

As seen above, the OnKeyDown callback on the ProxyKeyboardParser also calls the intercept_recording_key function, if the key press was one of the aforementioned command key scenarios.

This intercept_recording_key function saves the modifiers and key to the channels buffer, incrementing the active_channel_index by two, for every key pressed by the user. That way both the state of the eight modifier buttons are saved along with the key presses.

Implementing user story #3 – Being able to replay keystrokes

Replaying keystrokes can only happen when the user is not recording. If this is the case, the user replays by pressing either the left or right control key, together with one of the numbers one through nine on the numeric pad.

When this happens, the replay_channel function (lines 289-311) is called. A loop is going through each of the two-tuples of modifier and key:

  • First determining if the key is larger than zero. If this is not the case, the replay is stopped immediately – as a zero key indicates the end of a sequence of keystrokes.
  • Then the modifier byte of the report is set.
  • The key is pressed and
  • finally the key is released.
keyboard_out.set_modifiers(mods);
keyboard_out.press(key);
keyboard_out.release(key);

The state of the modifiers are saved (line 307) before and restored (line 291) after the channel is replayed to the computer.

Implementing user story #4 – Record and replay multiple sequences

This was actually implemented together with the user stories #2 and #3, as I found it easier to have this functionality implemented from the beginning, rather than implementing it simple and advancing from there.

The buffer, called channels, in which the keystrokes are recorded, is actually a two dimensional array, where the first index dictates which of nine channels the recording is to be stored in or replayed from. Every function manipulating keystroke data, takes an unsigned integer argument referencing which channel the operation is to be performed in.

This is again why the user presses control + shift + a number on the numeric pad, when starting a recording. The latter tells the device which channel the recording is going into, the same goes for the replay.

Implementing user story #5 – Storing sequences on a physical medium

The RFDI/NFC shield is connected and it is verified that the Arduino Leonardo can communicate with it (it uses an I²C bus). It had to be modified acording to this guide from Adafruit.com, removing the connection to pin 2 and re-routing it to pin 6.

As I have had an agile approach to this, estimated what I could get implemented in the project period and done my implementation one user story at a time, I didn’t get to implement the storing of keystroke data on an RFID MiFare tag. This was mainly due to trouble in the initial phase of finding compatible hardware and implementing a keyboard proxy, bypassing the Arduino’s anti-feature of translating ASCII characters to key strokes.

Implementing user story #6 – Securing the physical medium

This could be implemented using hashes generated from a users password as keys when storing data on the MiFare card. The user could enter this password as the first key sequence when inserting a new RFID tag in the device.

The only problem is that the MiFare’s encryption scheme is no longer considered save as hackers as early as 2008 discovered ways to reveal secrets off the tags in minutes using a regular desktop computer, see http://www.computerworld.com/article/2537817/security0/how-they-hacked-it–the-mifare-rfid-crack-explained.html.

One could employ a stronger encryption scheme, such as an algorithm in the AES family.

Further work

Below are suggestions for further work, for me or anyone reading this:

  • Implementing the two remaining user stories #5 and #6 as mentioned above,
  • Verifying the control sequence (ctrl + shift + # on numeric pad) is not used by other programs at the police,
  • verify that the relevant computers at the police force supports USB, alternatively implement the sending part of the device using the PS/2 keyboard protocol.

Concluding

I have build a device, using the Arduino platform that sits between a USB keyboard and a USB enabled PC, forwarding any key press, except the ones that are used to control the recording, stopping and replaying of key strokes on nine channels.

The device fulfils the need of a platform independent clipboard across multiple applications on a single computer, but the transfer of key strokes between machines using a physical medium remains.

I hope you enjoyed reading this article – please feel free to ping me on e-mail or Twitter (@kraenhansen), in case you have comments or questions.

One Response to “Arduino Project: Platform Independent Clipboard”

Leave a Reply