Update: August 3 – WebSockets, holiday, flu, Design Patterns and IP

Posted on August 3, 2011

0


Summary

Flu struck me this week. Other priorities – when returning from Italy to Holland – last week. Holiday the week for that.

In the holiday I studied Design Patterns, Intellectual Property and Open Source licences to increase my knowledge on two fields I know just a little about.

Just before holiday I finialized the first proof of concept for WebSockets, establishing a two way Socket connection with Remote Objects in JavaScript, running in Chrome.

Design Patterns

Whether I use them or not, to know what they are and what makes what pattern what is quite relevant I think. It will improve my own coding and the results you will download when you want to contribute on this project.

To assure I will own the knowledge I started to map them and write blog posts on Design Patterns.

Intellectual Property and Open Source Licenses

Choosing which Open Source License will apply will also determine how useful the code will be for others and how many people will not contribute.

The goal is to go for a licence that is as open as possible, allowing both open and closed projects and both commercial and non-commercial projects to happen, where the only restriction is that you respect the authors and the work itself.

Git or SVN?

Many projects I know have moved from SVN to Git repositories. Apparently Git is the new shit.

Instead of sticking to what I know, I decided to drop SVN for Git.

I have been considering maintaining two repositories, by mirroring the base libraries to a Git and SVN repository, but SVN for sure will freak out when files suddenly disappear while they are still registered in the local .svn lists. So Git it will be. And ZIP files for those who want to just download the code base.

WebSockets

As you might have read, I consider WebSockets to be of importance for the framework. First of all, because it allows you to build applications in the browser, without the need of any compiler or special IDE.

Second, WebSockets might help solve the issue of specific settings on Firewalls, blocking any traffic that is not HTTP or FTP.

I run some successful tests in Chrome with a simple chat and tracing system, showing me connection state and events and activity on the Browser side.

Next steps

I hope to pick up steam again in the next week. The total of three weeks not being productive on HotForestGreen was not foreseen.

The new deadlines are now set to:

  1. September 1st – Release of the C# and JavaScripr code base and several demos to get started
  2. October –

Older Status Updates

  • June 1: Prototype code for first phase of HotForestGreen almost done, implementing authentication
  • June 13: First round trip successful
  • June 19: Gettign a grip on all things considered. After a round of high-impact refactorings, things got to the point of functional again. Also contains a lot of background info on SandBoxes, the new security model, Systems of Trust and the revised Auto Connect system.
  • July 5: First tests with Remote Objects in the refactored code. After a thorough refactoring, I tested if the basis is still working.
  • Agust 3 – This post

What is this Framework about?

See the post of June 19. See also the “About” page here.

Update on roadmap

Phase 1: Building the basis – In progress per May 4, 2011

Before June 10, 2011 I hope to finish Phase 1, which covers the basics on data, data conversion and Client/Server and Client/Client messaging. This includes:

  1. Remote Objects – Sending and updating Remote Objects to all Remote clients
    1. Status: June 19, 2011
      1. Basis is tested last week. Found some bugs in retrieving persistant objects, leading to creation of news ones instead of retrieving the old
      2. Need to be retested. Will be done in the next 2 weeks.
    2. Status: July 5, 2011
      1. Bug: Echo from own data – Remote Objects seem to receive an echo from their own sent update. This is unwanted behavior.
      2. Bug: Application crashes – As Visual Studio rather crashes than show me which exception is not handled, it is hard to find what is going on. It is probably related to the Remote Objects.
      3. Test: Synchronized Browsers succesful – I have tested the Remote Objects in a simple experiment with synchronized browsers.
      4. Improvement of Event Model – The Remote Objects are receiving a more and more sophisticated Event Model, making it easier to monitor what is going on.
  2. Server feedback – Including IP numbers of Servers serving specific sandboxes
    1. Status: June 19, 2011
      1. Tested and in revision now. Refactored to make the message in standard BaseMessage format. Will do more refactoring today to revise the process of: “knock knock, who’s there?”
    2. Status: July 5, 2011 
      1. No progress.
  3. Heartbeats and auto-reconnect – Including auto-discovery of services and sandboxes and auto-reconnects when a Client connection drops or a Server drops out
    1. Status: June 19, 2011
      1. Revised to be independent of Servers running. To be tested in a Server Cluster.
    2. Status: July 5, 2011
      1. Bug fix: Static timer – The timer used for the heartbeat of the SandBoxes only dealt with one of all registered sandboxes, as it was a static instead of a local variable.
      2. Preparation for multi-tier servers – To support multiple port numbers the SandBox connector is updated. Tests today to make it work properly have been cancelled. To be picked up later as it is only a priority to test Server to Server communication.
      3. Test – Auto connect seems to work fine. Within 20 seconds of a Server becoming available, the separate applications are synchronizing Remote objects.
  4. User Authentication – Including a basic “plug and play” scenario for logging into SandBoxes and assuring messages sent from a Client are “signed” by the user Credentials
    1. Status: June 19, 2011
      1. No further progress
    2. Status: July 5, 2011
      1. No progress
  5. Data manipulation – Using a “plug and play” approach via: “Transparant Data Access Layers” to access, change and store data in databases
    1. Status: June 19, 2011
      1. Tested first version three weeks ago. To be revisited after finalizing items 1 to 4.
    2. Status: July 5, 2011
      1. No progress
  6. Data Conversion – Including the conversions of Objects to “Comma Separated Values” vice versa, in order to:
    1. Send updates – On Remote Objects through the network
    2. Reduce code for working with Databases – As we can inject data from databases into objects and database records
When these main items are done, I can release the code and start building some proof of concepts, showing the possibilities of the framework.

Phase 2: Securing the framework – Started per June 16

Phase 2 is about making the Framework more solid and more protected against people who want to hijack data and events.

  1. Server to Server updates – Including: “I am currently serving Sandboxes A, B and C with a total of N connections”
    1. Status: June 19, 2011
      1. Started implementation of Secured SandBoxes to make this possible and simple
      2. Implemented Server Sate Manifest as a Remote Object that contains the data per Server and automatically updates all other Servers when the state on that Server changes
    2. Status: July 5, 2011
      1. Bug fix: Secret sandboxes not registered properly – Due to a bug in the refactored code, allowing for the creation of three different sandboxes: Open/Public, Trusted/Public  and Hidden, the Internal Server Manafest was not sent and received properly. This bug has been fixed yesterday.
      2. Other progress – Briefly tested, except from the bug fix, no progress made
  2. Load Balancing – Including a smart distribution of Sandboxes so that the load to serve Clients is shared by many
    1. Status: June 19, 2011
      1. Implemented, not tested yet. When a Client requests an IP number, the Client Socket will provide the IP of the Server that is least loaded.
    2. Status: July 5, 2011
      1. No progress made
  3. Server to Server security – Using handshakes and shared keys. Assuring that when a user plugs in a “Server” to intercept messages and sandboxes, this “Server” is refused by the network
    1. Status: June 19, 2011
      1. Work in progress. Servers now build up a Secured SandBox using the exact same mechanism as Clients, with the difference that they use an internal Shared Keyword to test if they can Trust the other Server
    2. Status: July 5, 2011
      1. Trust relationships seem to work – The basic principle has been tested and trust-relationships seem to work. Connections are refused when a Client Server can not be trusted and three different type of SandBox Connections have been defined. See “Server to Server updates” under the update of July 5, 2011.
      2. More testing required – To be sure that the code is solid, more testing is required.
  4. SSL encryption – Currently all data sent is in Clear Text, making it extremely easy to hijack and abuse the system
    1. Status: June 19, 2011
      1. Not started yet
    2. Status: July 5, 2011
      1. Not started yet

When these items are done, the Framework can withstand most basic attacks, can be scaled up by hot-plugging new Servers in the network and will continue to work even when multiple servers drop out.

Phase 3: Opening up the framework – Started 

While the basis is for specific use in a controlled environment, some things are missing to make the framework even more useful. These include:

  1. The use of Web-sockets – For Socket-communication over port 80 in environments behind firewalls
    1. Status: August 3
      1. WebSocket Server is working for current implementation / protocol.
  2. Porting the code from C# to Java and ActionScript – To provide a broader use base, allowing you to run the framework on:
    1.  Android, Apple, Linux and in Flash
    2. Intel and ARM-based devices
Other possibly needed ports I will leave up to the community that will hopefully emerge. I simply do not have the time.

Phase 4: Testing and cleaning up – Not started yet

In Phase 4 all crap and remaining leaks will be filtered out of the system. During phase 2, most issues have been tested and addressed, but as a part of the process.

The work includes:

  1. Testing specific use-cases for performance – By running as much connections and messages as possible, until the system collapses
  2. Testing possible ways to hijack users and data – By scrutinizing the security model and implementing code that should break it
  3. Improving performance – By taking away data-hogs and
Advertisements
Posted in: update