We have described the general server design and threading model in our last article. Today we will go under the hood to explore the engineering details of how data is transferred between the server and the client. On the high level, instruction requests are harvested in sets and queued for processing by the worker threads. The worker threads resolves the XML instruction requests into a set of executable instructions on the server and processes them after correctly authenticating the source. The complexity of this system is based on a simple but advanced communication protocol designed for today’s needs.

Communication Protocol
The communication protocol between the server and the client is in XML. The client aggregates a list of instructions that they want executed on the server and then transmits that to the server. The sequence of the execution is determined by the ordering of the client’s XML request, which is done bottom up. The last instruction at the bottom of the XML request is the first to be executed on the server, followed by the last second and so on. The format of the protocol is not entirely XML based. The protocol divides all the data received or send into two segments. The first segment hold the XML instruction, the second segment hold raw attachments, if any. Attachment segment may hold additional file contents that are not XML based (eg. binary data). The first 8 bytes or 64 bits dictate the total data block size. The second 4 bytes or 32 bits indicates the first segment size, the remaining data belong to the second segment.

This protocol design although simple will allow us to provide a basic mechanism to transfer vast amount of differing types of data depending on the XML instruction request defined by the first segment. You can view the first segment as the instruction manual to process the request or sets of requests and the second segment as any other additional data required to fulfill the request or sets of requests. It may seem that this protocol is nothing special on first inspection but we think otherwise. Many of the protocols used today (eg. web server) are done using fixed Id values and headers which are less flexible and robust. The reason is that these older protocols mainly focuses on the format of transferring data and stopped there. Data that we communicate to each other today demands more than just being transferred but also properly processed to have any real value.

A simple example will be a file transfer using this more advanced high level protocol. The first XML segment can define a file transfer indicating how big the file size is in the attachment, the file name, destination location and etc. The protocol can also handle multiple file transfers in a single request. This will then allow the server to save the file to some predefined location. The XML segment might also define a decompression instruction that will result in the transferred file being uncompressed at the end of the entire exchange. The XML segment might also further define what is to be done with the uncompressed data and etc. This results in a more flexible and robust system that can be self descriptive and automated at a high level.

If you use our work, we ask that you remember to give us the credit that we deserve. Our hope is to develop better technology through open sharing so that future generations can benefit. Research and development for newer and better technology is still in great need as our generation moves towards the digital age, so join us in this fight towards the cutting edge of information technology.

We have been discussing among ourselves about open sourcing Hiitch for a while now. When we say open source Hiitch, we mean both the server and the client code. A totally complete and open desktop social networking platform that others can use, extend, customize and redistribute on their own. Can open source really help us compete or is it the end of us? We have only recently come to an agreement that open source is an option that we will seriously consider in the near future. We have put into Hiitch our heart & soul and we are unsure if open source is a right move at this stage.

Why do we want to open source Hiitch?

• We envision Hiitch to become a free but personal social networking platform that everyone will use to meet new friends and stay connected. No barriers!
• We want to move the development fast enough to challenge the current market players and existing technologies.
• We desire a vibrant development community that will keep innovating with us.

Our biggest problem now is that we do not have the financial resources to fulfill all or any of the above goals. My argument for open source is that we can allow others to build smaller networks of their immediate social circle and then build software to connect these smaller social networks together to form a mega network of independent networking peers. We can achieve this with little or no cost at all using open source approaches. The open source community has already shown that the last two points is workable so I will not elaborate more on it.

What are our fears of open sourcing Hiitch?

• It scares away potential investors and greatly reduces our value as a commercial company.
• The open source community do not see a need for such a product and hence no vibrant development community.

The worse thing to happen is that we cannot financially support ourselves to innovate and develop ground breaking software for the social networking messes. Merely selling value added services on top of a free software platform is also not an easy approach. We also worry that a geek community cannot attract enough normal average users for which a life style application like Hiitch is build for.

Feel free to share your views with us on this matter.