There is client-server software, and there is peer-to-peer software. Are these two architectures all we ever need for distributed software?

I’d like to suggest that some of the world’s most successful distributed software architectures are neither client-server nor peer-to-peer when you look at them closely. They follow an architectural pattern that I’d like to call the 4-Point Architecture.

Consider e-mail, a distributed and network-centric application if there ever was one. To accurately describe how e-mail gets sent from my computer to your computer, we need four computers in the picture (the "four points" of this architecture):

In this case, My Computer first hands the e-mail message to my SMTP server through one of several possible protocols (e.g. SMTP, POP, …). The SMTP server then uses SMTP to send the message to your SMTP server, which in turn holds the message until Your Computer downloads it using a protocol such as POP.

In this architecture, we have elements of both peer-to-peer and client-server: either SMTP server can initiate communication to another SMTP server, which makes them peer-to-peer. (However, SMTP is not a symmetrical protocol, which means we run client-server on top of peer-to-peer.) The relationship between an SMTP server and its client computer is clearly a client-server relationship.

If we redraw the picture in a more generic manner, we arrive at this: (and I will get to the labeling in a second)

The 4-Point Architecture is also used by the following technologies:

• Jabber/XMPP presence and instant messaging. Jabber clients log into Jabber servers using an asymmetrical protocol. Jabber servers talk to each other using a symmetrical protocol. Same picture.
• Modern P2P file sharing networks. It turns out that virtually all of them distinguish between regular nodes and super nodes. While the communication between nodes may allow for an entirely peer-to-peer form of communication, in practice only subsets of this protocol are used, making peer-super-peer relationships client-server and super-peer-super-peer relationships peer-to-peer. While dynamic reassignment of roles may take place, they still use the 4-Point Architecture at any point in time.
• Update April 25: Of course, blogs work the same way as well: we edit locally on our PCs but publish on visible servers with a defined address.
• There are lesser-known ones as well, including what we have in our product.

We could call this architecture a mix of client-server and peer-to-peer. However, it comes with some rules that cannot adequately be described without looking at all 4 points of the architecture:

• The computers in the first row must be addressable and routable by any other computer in the first row. As a result, they must belong to the "bright" internet (i.e. those parts of the internet that can be reliably addressed and found)
• No computer in the second row needs to be addressable and routable. To communicate, they must be able to initiate a connection to a bright point, but there is no requirement that a bright node initiate a connection on its own. Therefore, the computers in the second row may belong to the "dark" internet (i.e. those parts of the internet that may have rapidly changing IP addresses, are moved quickly from network to network, such as laptops, and generally do not have a well-defined DNS name).
• Any dark point only interacts with exactly one bright point (at least for a given application or protocol). It is extremely rare that a dark point interacts with more than one bright point (for a given protocol) for reasons other than basic availability of bright points. Thus the relationship between a bright point and its dark points is a fairly stable one.
• No dark point ever interacts directly with another dark point, they always go through their respective bright points first.

Why do I think this 4-Point Architecture is important? It’s important because it is a generally-useful, proven architecture for distributed applications that includes all kinds of devices, not just "bright" servers. Some of its benefits are:

• It clearly assigns responsibilities: bright points must be stable and available, for example, while dark points need not be.
• It enables reliable communication between unreliable dark points that may not even be able to route to each other directly, regardless how much work one was willing to put in.
• It allows organizations to specialize: operating a bright point requires a different set of expertise than operating a dark point.
• It allows local innovation: if somebody invents IMAP, they can install it locally (between their bright and dark points), without impacting the rest of the network.
• It provides clear guidance where to attach logging, approval, quota and so forth functionalities and procedures.
• It reduces the requirements on the dark nodes, which are often relatively underpowered devices (e.g. cell phones can send e-mail without having to support the entire SMTP protocol).

Watch the 4-Point Architecture to become more prominent going forward in many places … it simply makes too much sense not to.

Yesterday’s personal news highlight, for me, was that:

…a bunch of computer-generated gibberish masquerading as an academic paper has been accepted at a scientific conference … the World Multi-Conference on Systemics, Cybernetics and Informatics

as reported by CNN. This relates to me, for two reasons:

My former boss Jürgen Bortolazzi, a prolific author of (good!) research papers, used to joke that it wasn’t really him who wrote those papers but that he had developed a Prolog-based paper generator. It’s great to see that somebody actually developed such a thing, and it works!

But closer to home, I know that conference! I had a paper there, a few years back, in fact even an invited paper! So I decided to travel to Florida and give a talk on it. And having done so, I remember very well that I turned around to my wife, who came with me for a brief vacation afterwards, saying: "what a waste of time" … maybe this was just the first known computer-generated paper at that conference.

I was laughing so hard yesterday, it simply had to happen, and it had to happen to that conference.

In the olde days, parents would teach their kids when and how to take off their hat, and how to address a town notable. In the early days of the internet, we got netiquette, primarily focused on the new social technologies of e-mail and and bulletin boards, although it now seems to be largely forgotten (maybe we all have internalized the rules?) With the recent explosion of all sorts of new social technologies (from blogs over social networking to Bluejacking …), most of the old techniques don’t quite fit any more and new ones are needed.

Yesterday, I met face-to-face with Jeff Nolan, whose blog I had been reading for some time. As it turned out, he had been reading mine as well. But I did not know that. Blogs create this funny situation where two people may know quite a lot about each other, without either having any idea about the other’s knowledge about themselves. Or having a common acquaintance, or background of any kind. Even if they quote each other occasionally, it may be because they just happened to do a referrer search and read something interesting on a blog that they will never again look at, or they may be genuinely interested in the person behind the blog they quote.

I’m pondering the need for additional social protocols given these new circumstances, even if it is straightforward with somebody like Jeff who is easygoing and part of the same industry and in the same location. It occurred to me yesterday because it was the first time I really had no prior interaction or intermediary …

But in the general case, how do you start a conversation with somebody you know through a blog? "How do you do? Have you solved the XYZ problem?" is not it, even one knows quite a lot about the subject due to the blog. Even the traditional "I’ve heard quite a lot about you" does not work any more because the "heard" requires an intermediary which does not exist either… Or maybe we just need to get used to falling right into the middle of a detailed conversation with people we have never talked to, or e-mailed, or interacted 1-1 before. But then, we don’t know whether or not we can. It’s a weird new world … what am I going to teach my 6-yr old about how to be polite? I guess I can drop the bit about the hat, but that’s where certainty ends …