Update: July 5 – First tests with Remote Objects in the refactored code

Posted on July 5, 2011



Due to other priorities I wans not able to do much since June 19.

Today I did two things:

  1. Test with synchronized Browsers – This test included the following elements:
    1. Synchronization of URLs – When one of the browsers navigates to another URL, all other browsers follow.
    2. Synchronization of scrolling – When one browser scrolls up and down the page, all other browsers follow.
  2. Fix of bug in heartbeat – I removed a bug in the heart-beat of the Application SandBox. This lead to only one sandbox scanning and updating connections  when no connections were provided.

The synchronized browser is one of the first proofs of concept and – as a project – will be cleaned up for reference.

Primary target: make implementation of Synchronized Objects as easy as possible

HotForestGreen is all about Synchronized (Remote) Objects.

Building this framework, I strive to create a solution that takes all the hard way out of the process, and only leaves the very basal things left for the coders using HotForestGreen.

This is slowly becoming reality. Each next iteration (there were 3 major ones in the past 2 months) makes it simpler to understand, without having to know the exact mechanics.

30 minutes

At the end of the ride, novice programmers should be able to have their first project with Distributed (Remote) Objects up and running in 30 minutes:

  1. Changing colors of screens of other computers
  2. Browsing web pages simultaneous on multiple screens
  3. Turning leds on and off on Arduino boards and via XBee modules by addressing (setting values on) Virtual Objects or Mediators

Open for improvement

  1. Layering applications over multiple ports fails – In the period of June 19 I implemented the options for multi-layered applications, living on several ports “on top of each other”. The idea is a bit like Flatland, where several universes can exist on top of each other. There are some bugs in the connection scheme which – right now – make connections from a Sandbox to one port fail if another is connected.
  2. Remote Objects seem te receive an echo of their data – When testing the Synchronized Browser, the Browser Object who sends an update to the Application Cluster seems to receive an echo of that update from the Server. This echo should not happen, as the object should be excluded from the recipient list.

Next steps

Since I was not able to do anything since June 19, this list has not changed.

In the next weeks I will:

  1. Finalize the Security layer, part of Phase 2
  2. Test, debug and clean up the prototype code for Phase 1 and 2.
  3. Test and debug the framework with some basic applications

Main focus for the next days

The main focus in the following days is on the following:
  1. Multi-tier Client Server connections – Mainly needed for Server to Server tests. As only one Server can open a Socket Listener to one specific port, I need to be able to listen to multiple ports and run the application over these multiple tiers.
  2. Remove the echo from Remote Object Updates – Right now, Remote Objects seem to receive an acho from their own updates to the Application Cluster
  3. Improve the Event Model for debugging purposes – I added some improvements in the Event Model for Remote Objects and will extend that for Servers and Connections as well. The main purpose is to make monitoring of events easier. A good event model beats step by step tracing in debug mode when you want to know where things went wrong.

The goal is to have something demo-able before August.

Finalizing Database access and working with databases is moved – for now – after getting the basic framework running and tested to satisfaction.

Older Status Updates

  • June 1: Prototype code for first phase of HotForestGreen almost done, implementing authentication
  • June 13: First round trip successful
  • June 19: Gettign a grip on all things considered. After a round of high-impact refactorings, things got to the point of functional again. Also contains a lot of background info on SandBoxes, the new security model, Systems of Trust and the revised Auto Connect system.
  • July 5: This post

What is this Framework about?

See the post of June 19. See also the “About” page here.

Update on roadmap

Phase 1: Building the basis – In progress per May 4, 2011

Before June 10, 2011 I hope to finish Phase 1, which covers the basics on data, data conversion and Client/Server and Client/Client messaging. This includes:

  1. Remote Objects – Sending and updating Remote Objects to all Remote clients
    1. Status: June 19, 2011
      1. Basis is tested last week. Found some bugs in retrieving persistant objects, leading to creation of news ones instead of retrieving the old
      2. Need to be retested. Will be done in the next 2 weeks.
    2. Status: July 5, 2011
      1. Bug: Echo from own data – Remote Objects seem to receive an echo from their own sent update. This is unwanted behavior.
      2. Bug: Application crashes – As Visual Studio rather crashes than show me which exception is not handled, it is hard to find what is going on. It is probably related to the Remote Objects.
      3. Test: Synchronized Browsers succesful – I have tested the Remote Objects in a simple experiment with synchronized browsers.
      4. Improvement of Event Model – The Remote Objects are receiving a more and more sophisticated Event Model, making it easier to monitor what is going on.
  2. Server feedback – Including IP numbers of Servers serving specific sandboxes
    1. Status: June 19, 2011
      1. Tested and in revision now. Refactored to make the message in standard BaseMessage format. Will do more refactoring today to revise the process of: “knock knock, who’s there?”
    2. Status: July 5, 2011 
      1. No progress.
  3. Heartbeats and auto-reconnect – Including auto-discovery of services and sandboxes and auto-reconnects when a Client connection drops or a Server drops out
    1. Status: June 19, 2011
      1. Revised to be independent of Servers running. To be tested in a Server Cluster.
    2. Status: July 5, 2011
      1. Bug fix: Static timer – The timer used for the heartbeat of the SandBoxes only dealt with one of all registered sandboxes, as it was a static instead of a local variable.
      2. Preparation for multi-tier servers – To support multiple port numbers the SandBox connector is updated. Tests today to make it work properly have been cancelled. To be picked up later as it is only a priority to test Server to Server communication.
      3. Test – Auto connect seems to work fine. Within 20 seconds of a Server becoming available, the separate applications are synchronizing Remote objects.
  4. User Authentication – Including a basic “plug and play” scenario for logging into SandBoxes and assuring messages sent from a Client are “signed” by the user Credentials
    1. Status: June 19, 2011
      1. No further progress
    2. Status: July 5, 2011
      1. No progress
  5. Data manipulation – Using a “plug and play” approach via: “Transparant Data Access Layers” to access, change and store data in databases
    1. Status: June 19, 2011
      1. Tested first version three weeks ago. To be revisited after finalizing items 1 to 4.
    2. Status: July 5, 2011
      1. No progress
  6. Data Conversion – Including the conversions of Objects to “Comma Separated Values” vice versa, in order to:
    1. Send updates – On Remote Objects through the network
    2. Reduce code for working with Databases – As we can inject data from databases into objects and database records
When these main items are done, I can release the code and start building some proof of concepts, showing the possibilities of the framework.

Phase 2: Securing the framework – Started per June 16

Phase 2 is about making the Framework more solid and more protected against people who want to hijack data and events.

  1. Server to Server updates – Including: “I am currently serving Sandboxes A, B and C with a total of N connections”
    1. Status: June 19, 2011
      1. Started implementation of Secured SandBoxes to make this possible and simple
      2. Implemented Server Sate Manifest as a Remote Object that contains the data per Server and automatically updates all other Servers when the state on that Server changes
    2. Status: July 5, 2011
      1. Bug fix: Secret sandboxes not registered properly – Due to a bug in the refactored code, allowing for the creation of three different sandboxes: Open/Public, Trusted/Public  and Hidden, the Internal Server Manafest was not sent and received properly. This bug has been fixed yesterday.
      2. Other progress – Briefly tested, except from the bug fix, no progress made
  2. Load Balancing – Including a smart distribution of Sandboxes so that the load to serve Clients is shared by many
    1. Status: June 19, 2011
      1. Implemented, not tested yet. When a Client requests an IP number, the Client Socket will provide the IP of the Server that is least loaded.
    2. Status: July 5, 2011
      1. No progress made
  3. Server to Server security – Using handshakes and shared keys. Assuring that when a user plugs in a “Server” to intercept messages and sandboxes, this “Server” is refused by the network
    1. Status: June 19, 2011
      1. Work in progress. Servers now build up a Secured SandBox using the exact same mechanism as Clients, with the difference that they use an internal Shared Keyword to test if they can Trust the other Server
    2. Status: July 5, 2011
      1. Trust relationships seem to work – The basic principle has been tested and trust-relationships seem to work. Connections are refused when a Client Server can not be trusted and three different type of SandBox Connections have been defined. See “Server to Server updates” under the update of July 5, 2011.
      2. More testing required – To be sure that the code is solid, more testing is required.
  4. SSL encryption – Currently all data sent is in Clear Text, making it extremely easy to hijack and abuse the system
    1. Status: June 19, 2011
      1. Not started yet
    2. Status: July 5, 2011
      1. Not started yet

When these items are done, the Framework can withstand most basic attacks, can be scaled up by hot-plugging new Servers in the network and will continue to work even when multiple servers drop out.

Phase 3: Opening up the framework – Not started yet

While the basis is for specific use in a controlled environment, some things are missing to make the framework even more useful. These include:

  1. The use of Web-sockets – For Socket-communication over port 80 in environments behind firewalls
  2. Porting the code from C# to Java and ActionScript – To provide a broader use base, allowing you to run the framework on:
    1.  Android, Apple, Linux and in Flash
    2. Intel and ARM-based devices
Other possibly needed ports I will leave up to the community that will hopefully emerge. I simply do not have the time.

Phase 4: Testing and cleaning up – Not started yet

In Phase 4 all crap and remaining leaks will be filtered out of the system. During phase 2, most issues have been tested and addressed, but as a part of the process.

The work includes:

  1. Testing specific use-cases for performance – By running as much connections and messages as possible, until the system collapses
  2. Testing possible ways to hijack users and data – By scrutinizing the security model and implementing code that should break it
  3. Improving performance – By taking away data-hogs and
Posted in: Uncategorized