TechDoc: The Automated HotForestGreen Server Discovery Process in 3 steps

Basics of auto-detection

Any basic HotForestGreen Client does auto-detection when “switched on”. This Auto Detection scans the local network the Client is connected to (via the device it runs on and the network that device is connected to).

It will automatically connect to an Application Cluster once it:

1. Has found one or more Servers.
2. Is allowed to join.

Images

Below you will find four images with some extra explanations visualizing the process and the full implementation.

Purpose

The purpose of the Automated Process is to:

1. Plug and Play – Instead of you having to do a lot of configuration on Networks and IP Addresses, the Framework will take care of that. Allowing you to focus your energy on what is relevant for you.
2. Simplify building Distributed Applications – To make building Distributed Applications, running over multiple machines, and within one or more Application Clusters, containing it all, as easy as cake.
3. Simplify connecting to Distributed Applications – To allow you to build (Stray) Clients to your Distributed Applications and work with whatever goes on inside.
4. Allow for fail safe- and fallback scenarios – By allowing you to add any number of Servers within your Local Network and do automated Load Balancing and automated Re-routing of communication streams when Servers drop out / crash

Targets

The targets for auto-connection are:

1. Servers and Server Clusters – Finding each other and communicating work load and number of connections
2. Trusted Clients – Running inside the network and the Application Cluster and being known
3. Stray Clients – Like mobile phones and tablets hooking into the Network and coming from anywhere

Scope: Local Network

This article covers the auto-discovery process within a local network.

Connecting to specific IP addresses online

This post only covers the Local Network. A separate post will cover the connection process to specific IP Addresses.

The process

Below you will find the connection process in three simple steps.

Using a step by step elimination process, from a wide range of “all IP Addresses in a specific range” you come to connect to one or more specific Servers, based on availability and acceptance. Here is a brief overview:

1. Step 1: Ping the entire local network – All 256 IP Addresses in a specific range are pinged.
2. Step 2: Who is a Server? Request Server Manifest – To all IP Addresses that  responded, a Server Manifest is requested
3. Step 3: Attempting to connect to the found Servers – As not all Servers accept just anything, some might refuse an attempt to connect. The ones that allow you to connect will maintain a persistent Socket Connection until kingdom comes, or the connection is interrupted.

Step 1: Pinging the entire local Network

Pinging the local network

1. Client IP – Based on the IP address the Client is connected to, the IP range is determined in which the Client could find the Servers. This can be one or more IP addresses, depending on how the computer is connected.
2. Simulatious ping – Per IP range a simultanious ping is done to all numbers inside that range, including the machine itself.
3. Silence and response – Each IP address will either respond on the ping, or not respond at all.
1. Response – There is a machine on the other side. We do not know yet if it is a Server.
2. No response – There is nothing there.

Step 2: Who is a Server? Requesting the Server Manifest

Sending the Server Manifest Request

The next step is to find out who of our present machines is a Server or not. We do this as follows:

1. Get Server Manifest – To each IP address (that responded on the Ping of step 1) a request is sent for a Server Manifest.
2. Silence and Response – Each machine that runs a HotForestGreen Server will respond with a Server Manifest. All machines which do not run a HotForestGreen Server will simply not be able to accept the attempt to connect from the Client

Step 3: Attempting to Connect to the Found Servers

Connecting to the Found Servers

For each Server we found in Step 2, we will attempt and receive the following:

1. Connect – The Client will not attempt to connect to the machines / IP Addresses that responded with a Server Manifest.
2. Refusal and Acceptance – Based on the security levels set on the Server, the Server might expect the client to provide a specific handshake.
1. Accepted – Either the Server and Client share the same secret keywords, or are open to anyone. A persistent Socket Connection is created and maintained as long a s possible.
2. Refused – Something did not match in the secret handshake between Client and Server. The Server or Client refuses to connect and no connection is established.

End result of the Discovery and Elimination process

At the end of the Discovery and Elimination process starting with all IP addresses, filtering out:

1. No response – All who give the silence of “no-one here
2. Not a Server – Then all who are not a Server
3. Not trusted – Then who are not trusted or do not trust you

If all went well we are now connected to one or more Servers in the Network, able to send and receive real time data to other connected Clients in that Network.

Time needed: less than 1 second

The whole Discovery and Elimination Process is time-optimized and can take place in less than 1 second, as all 256 IP numbers are scanned simultaneously.

Saving more system resources

As most Servers will probably linger in the IP range from xxx.xxx.xxx.001 to – let’s say: xxx.xxx.xxx.o50, due to the way DNS numbers are issued to “who comes first gets the lowest number”, and most Servers where there before most Clients, a lot of overhead can be saved by scanning IP ranges in blocks of 20 to 40 successive addresses.

If this improvement will be implemented depends on negative experiences while running the framework and possible future developments to optimize this process even more.

Plug and play

No human interaction was needed:

1. To find and define the IP address(es) of the found Server(s) – as this was derived from the IP address assigned to the computer when it entered the network
2. To find and to connect to the Application Cluster – As the load-balancing and the internal handshakes take care of connecting Clients to SandBoxes and exchange data within these SandBoxes between the Clients.

Once connected: Running multiple Sandboxes, Servers, Ports and Layers

The auto-discovery process as shown above is only part of the whole story. In the image below I attempted to show all the options you have after a Server is discovered.

They will occur:

1. When you have multiple Servers – Arranging Load Balancing and Auto-wiring you to the proper Server, serving your requested SandBox
2. Running your application over multiple ports – When you connect to other Ports in your network.

Why is this relevant?

1. Multiple SandBoxes: separating data streams – You will have data you want to separate from others, to prevent data-leakage, data contamination (as you run Remote Objects with the same ID, but different objects attached to it) or simply to secure and shield specific data streams.
2. Multiple Servers: reducing dependencies on one single machine – Once your application has to be more robust, you want to have fall back scenarios for when a Server drops out / crashes.
3. Multiple ports: running multiple Servers on one machine – This started as a solution to allow me to test several Servers running simultaneously: sending Server State Manifests to each other to allow load balancing. As it is in place anyway, it has become part of the framework at large, for you to use. Leading to the possibility to create multiple Layers.
4. Multiple Layers: separating streams even more – Your public services will run on one generic, default port, available to anyone to hook into. But some data running over your network, you want to keep private. Even more than possible via Secured SandBoxes. By running specific Servers over Specific Ports and using Trusted Clients as mediators between those layers, you are able to tap into both, exposing limited access to Stray Clients and hiding full access on some secret port.

Multiple SandBoxes, Servers and Layers visualized

Multiple Sanboxes, Servers, Ports and Layers

1.  Multiple Sandboxes over Multiple Servers – HotForestGreen Servers within the same network are interconnected and share their work-load. So when you connect to “a” Server, you might actually be re-directed to another IP address which is best to serve you for that specific SandBox.
1. Multiple Servers – The moment you need a more robust solution with fall-back scenarios in case of Server Failure, you will have two or more Servers in your network. All of these Servers communicate to each other, sharing their current state and work load in real time.
2. Multiple SandBoxes – Each SandBox is a closed environment in which data is protected and shielded from accidental contamination. You will find that you will – soon enough – run two or more SandBoxes to host the data streams related to your application.
2. Multiple Layers by using Multiple Ports – One Cluster running over one single port over multiple machines can be considered one “Layer”.
1. Layers – A Layer is like a sheet of paper on which your network draws its data.
2. Separating Layers – It is possible you want to run separate Layers and create “hidden layers” for specific tasks. For instance to separate your common and public data streams from the one that are more sensitive.
3. Ports – A port is like a channel on which your broadcast your data. When you connect to that port, you will be able to receive the data. If you do not, the data will be invisible for you.
4. Clients and Servers running on one Layer – Are able to discover each other and exchange information.
5. Clients and Servers running on multiple Layers – Can only see and discover other LAyers by explicitly connecting to it.

Fail saves

The system has several fail saves built in for when machines drop out of the Network.

Heartbeat and connection attempts

When a Server is not up yet, the Client falls back on a constant heart beat of (by default) 20 seconds, in which all connections of all Sandboxes are checked. If no connection is there where it should be, the Client will attempt to (re) connect to a Server.

Load balancing: pointing Clients to the right direction

While Load Balancing might – at first – not be perceived that way, it is (and allows for) a fail save.

Here is how:

Each Server in the Network is aware of the State of each other Server. So when one Server drops out, all other Servers will know almost immediately and will be able to define which Server is most fit to jump in the gap and take over the services it provided.

When clients, who were connected to the disappeared Server, start requesting for new a new connection for a specific SandBox, the Load Balancing takes care of pointing them to the proper new Server that will handle their requests.

Client- and Server side Caching: Preventing loss of real time data

In this first version of the framework, data loss due to a Server Dropout, is not covered.

This will be covered by Server and Client side Data Caching. When Clients lose a connection, the data for that specific SandBox(es) can be buffered until a new connection is established, making sure no data is lost.

On the Server side, data received can be cached for all Clients who come in later: updating them on the events that happened in the past N seconds.

Automated re-wiring of data connections

This allows for scenarios where Servers drop out, crash or die, where Servers kick in later, or Servers are replaced/hot swapped. All while the application at large continues to function and run with no visible effect apart from the small hiccups when Clients and Servers start actively re-discovering and re-wiring their connections to other Servers.

Posted in: techdoc