TechDoc: WebSockets and HTML based Web Clients

Posted on July 9, 2011



There is more to HotForestGreen than Java, Flash and C#.

As I started working on the (new) road map for this Framework in 2010, Web Sockets (and a simple integrated HTTP Web Server) became a very important part of it.

Why WebSockets?

Web Sockets in a Web Client, connected to a Server

There are several reasons for the implementation of Web Sockets:

  1. Allow for communication where Low Level Sockets might not be – In some environments, anything that is not a specific Socket with a specific protocol (in most cases: “is not HTTP”) is blocked.
  2. HTML-based Web Clients – Which can be build by anyone with a text editor and allows for quick development and a very low point of entry for anyone who wants to start playing with HotForestGreen.
  3. No compiler needed – As they run in JavaScript inside HTML pages inside a browser, the only thing you need is a browser that supports Web Sockets
  4. Can be used by anyone – With a basic knowledge of HTML and JavaScript
  5. Allow for a simple distribution model: the web – With any non-web based standard, your application needs to be downloaded and installed to work. Wwith Web Sockets, the only thing your users have to do is connect to the right location, with a browser that supports Web Sockets

HTML based Web Clients

Web Clients can be of any type including: Flash, Java and HTML.

HTML based Web Clients are applications that run (in most cases) in a Web Browser, communicate to a Server and use HTML (and JavaScript) to process and present data and interact with the user and the Server.

Useful links on WebSockets

  1. jWebsocket for AndroidThis article by Predrag Stojadinovic describes how the WebSoeckets and handshakes are set up

The ABC of a WebSocket

Let’s assume we have a web socket connection like this:

var host = "ws://localhost:80/mySandBox?myVar1=1&myVar2=2";
var ws = new WebSocket(host);


A WebSocket Header will be sent on connect and can contain the following base data, separated by a LF or CR/LF.

The header as received on the Server can look like this:

GET /test?x=1&y=2 HTTP/1.1\r\nUpgrade: WebSocket\r\nConnection: Upgrade\r\nHost: localhost:821\r\nOrigin: null\r\nSec-WebSocket-Protocol: someprotocol\r\nSec-WebSocket-Key1: &q 19 6 54 c6 C& & 533 <a7 (\r\nSec-WebSocket-Key2: 27W 72 00 0x5 94\r\n\r\n����J��g” object {string}

When we replace the \r\n for enters we get this:

GET /mySandBox?myVar1=1&myVar2=2 HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: localhost:821
Origin: null
Sec-WebSocket-Protocol: null
Sec-WebSocket-Key1: &q 19 6 54 c6 C& & 533  <a7  (\r\nSec-WebSocket-Key2: 27W  72  00 0x5 94
\r\n\r\n<8 byte long binary data for key 3>

Connecting to a SandBox

If you look at the red part of the URL, you will see the sandbox name:


The requested SandBox is passed to the Server as if it is a folder on that Server.

Web Socket Server Response

The Server Response we send back is mostly an echo from what we received, shuffled around.

For protocol V76 we add the secret keys and a check sum in our response, which is based on the three keys we sent.

HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Origin: http://null
Sec-WebSocket-Location: ws://localhost:821/mySandBox


WebSocket Header Parts

Below is a full overview of the header, with a short overview of each part. As we only use a very small sub set of the call we receive (marked in red), all parts we do not use for the framework will be marked grey. The parts vital for the returning hand-shake are marked green. Other parameters used in the return set to the browser include our red ones and the ones marked blue.

  1. GET/POST – The beginning string of our header reflecting the way the data is sent
    1. URI – The part right after our base connection information. In our example this is: “/mySandBox?myVar1=1&myVar2=2”
    2. HTTP version – The verison of the HTTP client/call. Unused
  2. Upgrade: WebSocket – The type of connection. We use this as the identifier for the Type of Socket Handler
  3. Connection: Upgrade – The type of connection as defined in Upgrade
  4. Host: localhost:80 – Our IP and port number to which we connected. As we already known this from our Socket Connection, we do not need this.
  5. Origin: null – Unknown purpose. Might contain the IP of the caller
  6. Sec-WebSocket-Protocol: <your definition> – Contains the protocol as you set it in the second variable when you create a WebSocket. Unused for HotForestGreen
  7. Sec-WebSocket-Key1: <a key value> – Contains the first web socket key from the client
  8. Sec-WebSocket-Key2: <a key value> – Contains the second web socket key from the client
  9. 8 byte binary data for key 3 – Contains the 3rd key, to be read as binary

Everything is binary data over a Standard Socket

When you dive into Web Sockets and how a Web Socket Server is created, at the end of the ride you will find a standard issue Data Socket.

Even the data itself is standard data sent as a binary stream over a binary connection

What differs from a “normal” socket is the following:

  1. The header – With each connection attempt from a Client to a Server, the Client sends a HTTP WebSocket header with specific data to establish a hand schake
  2. The hand shake – The Client expects a specific reply from the Server, to establish a low level type of trust: that it is connected to a Web Socket Server
  3. Information about packages and (in case of binary data) package size – For Web Sockets, these are mainly used for binary data
  4. Start and End Bytes – Each data package we send to a Server or Client starts with a Start byte (<0x00>) and ends with an End byte (<0xFF>)
Everything happening in between can be – in principle – completely binary, using – in most cases – 8 bit bytes.

Everything is Clear Text data with specific separators

HotForestGreen works primarily with Clear Text Data, separated into specific blocks using specific ASCII codes.

The main reason is to keep things open and simple when we translate and de-serialize the incoming packages.

If you need to send binary data…

Where binary data is required, we send two packages:

  1. Header package – A header package indicating the Server to expect a binary stream of a specific length
  2. Binary stream – The binary stream itself.
Right now (July 2011) the implementation for binary data is completely rudimentary and does not work. There is no priority yet to implement this.

WebSocket Start and end bytes

Each WebSocket data package is wrapped in a Start and End Byte. Like this: “<StartByte>My Socket Message<EndByte>”.

More concrete, these bytes are: “<0x00>My message<0xFF>”.

Network Streams and WebSocket Messages

A WebSocket Client requres an HTTP complient protocol to send Socket Data to thee client.

For this we use a (C#) NetworkStream instead of a bare Socket.

These streams state their lenght in the first parts of the data stream and as such have a slightly different build up from a generic Socket Connection which basically just streams data when there is data to stream and has no terminator at the end of the stream – except for 0x00 values as there is no data in the buffer used to inject the Socket Data in – when done writing.

Limited implementation for Web Sockets

WebSockets only have a limited implementation due to their character of running in an open space and using JavaScript code exposed and retrievable as clear text.

  1. No auto connect – You need to specify the IP Address to which you want to connect via a Web socket
  2. No Trusted or Secured Sandboxes – As these require a shared key on Client and Server and as HTML is uncompiled clear text, we rather not.
  3. No binary data – As stated before, this is not a priority yet, as we have no use cases yet where Web Clients need to send binary data to an Application Cluster

No auto connect

The auto detection process as used on the C# and Java Clients runs in several threads, to allow for parallel scans over multiple ports. If done in a sequential order, the process might take several seconds to complete.

Short and simple: I wanted to keep things simple and avoided this for Web Sockets

Scope of Web Sockets

WebSocket Clients are designed for use in:

  1. HTML clients – To allow for simple and fast designs and simple distribution of applications using the Real Time Interaction models of HotForestGreen over the web
  2. C#, Flash and Java clients – To connect to Web Socket based Application Access Points
  3. Public Sandboxes – Which do not require a secret hand shake and accept any client trying to connect

Public SandBoxes – and what they are

There are three types of SandBoxes:

  1. Public SandBoxes – Allowing anyone to connect, if their header and stated protocol is recognized
  2. Public Trusted SandBoxes – Allowing only Trusted Clients to connect: using a secret handshake with passwords both Trusted Client and Trusted Server know. If either of them fails to give the proper response, no Trust can be established and the connection is broken.
  3. Secured SandBoxes – These are SandBoxes hidden under a specific key and secured via a Trust Relationship. For instance: the secured Sandbox: “mySecuredSandBox” with key: “my secret key” will be hidden under “my secret key::mySecuredSandBox”. This makes scanning and attempting to connect to Secured SandBoxes a bit harder.
Posted in: Uncategorized