June 20, 2020

An Open Content Exchange Standard

uprotocol Banner

#architecture  #future-projects  #uprotocol

Let's create an open standard enabling users and developers to do more with what they have!

Every year, computer hardware is becoming more powerful, which is a great thing to see. However, this also means the software it uses needs to utilize its full power, which may become a challenge when the software uses a proprietary protocol that blocks users from improving.

Most content exchange protocols are an example of this because none of them are open for other solutions to use, causing inconsistency when users are out of the ecosystem they support.

An open protocol can fix this issue by enabling developers to create new solutions and allowing these solutions to communicate with each other.

What is the problem?

Existing content exchange protocols are not open and nearly impossible for third-party solutions to use because their developers don't document the way they work, and they usually are proprietary with no source code available.

Apple's AirDrop and other similar solutions will work as long as you are using them in the environment that their developers intended. You are out of luck if you use AirDrop to send a file to a device running ShareIt because they will not recognize each other. If you were to create a new solution that will work with one of these, you would need to log the communication process and replicate it, which is illegal and not intuitive.

What may be the reason for this disconnectedness?

This problem doesn't gain any traction because everyone has their number one solution for transferring content, and it works for them. You send A to B with C, and when C is missing, you install it, and if C is not available in any way, you do D.

If you ask people what they use to solve the content exchange problem, you will receive many replies suggesting a different approach, but they will never recommend a solution that does it all. They will say this has this thing missing, and that doesn't work on that and similar.

We don't pay too much attention to this inconvenience because we know one solution will work. However, we forget it takes more time to do a simple thing when the solutions keep failing.

What will this fix?

If developers adopt an open protocol, users will not need to use a specific solution. They will be able to use any solution made by anyone. After all, the solutions they choose will have little importance in the case of transferring files. They will have the freedom to pick any solution based on their preferred platform and software quality.

What do users gain?

Most people have limited resources when it comes to paid services. The internet may be limited or slow; they may not have the proper hardware or the configuration, or they may be in a place where their existing solutions don't work.

Because most solutions are nowadays proprietary, it looks as though the need for this type of protocol is unnecessary. However, this view has proved itself wrong after Xiaomi, OPPO and Vivo launched Peer-to-Peer Transmission Alliance1 for better compatibility between their products. But they didn't post the details for the protocol they use, which shows somebody has to take the lead and do it for the community.

With an open specification, users won't need to wait for an alliance to include their feature requests. They can join the discussion at any time.

Why not existing protocols?

Existing protocols—FTP, BitTorrent, SAMBA, and others—are not end-user friendly. They need a central server configured beforehand. In the case of BitTorrent, it needs trackers distributing peers.

Another problem is these protocols are trust-oriented. FTP and SAMBA have users, paths, and permissions to manage. For a better user experience, servers and clients should be unaware of each other before introduction and become acquainted after the first communication.

Isn't there an existing protocol addressing this problem?

BitTorrent is near perfect, but it is too far away from the experience I would like to have with TrebleShot and other solutions in general. It doesn't have a validation process for the clients, and adding on top of it may need unnecessary work.

Teleport2 defines a simple diagram with the HTTP protocol on top. But it is too simple to be useful, and HTTP may be unnecessarily over-bloated for a task like this. The new standard should be easy to port to different OSes and frameworks. The base protocol should at least be related to file operations similar to FTP and BitTorrent.

Development process

Because many things are undecided at the moment, everything is open for discussion. I have experimented with TrebleShot and created CoolSocket3, a TCP messaging library, but I haven't looked into other standards that may be better.

If a proper protocol surfaces before the development begins, we can build on top of it. However, there seems to be none at the moment.

I have created the GitHub organization for the new protocol called Universal Content Transportation Protocol4 or uprotocol for short. The names are also open for discussion.

I hope that the protocol will take shape in the coming months, and we will be able to implement it in apps like TrebleShot soon. The specs will be defined and showcased in the specs5 repo in the uprotocol organization.

If you are interested, create a discussion by opening an issue.

  1. Peer-to-Peer Transmission Alliance, Xiaomi, OPPO, and Vivo partner to bring new wireless file transfer system to global users. 

  2. Teleport, send files on the local network. 

  3. CoolSocket, easy-to-use multi-platform TCP communication layer. 

  4. Universal Content Transportation Protocol, a protocol specification for peer-to-peer content exchange that aims to unify existing solutions. 

  5. uprotocol/specs, Protocol details, proposals, and other helpful documents. 

Series 2.0

This post is part of a series. This section shows other posts in the same series.

TrebleShot  /  Completed

uprotocol Banner

June 20, 2020

An Open Content Exchange Standard

Let's create an open standard enabling users and developers to do more with what they have!

Model — View — ViewModel

Feb. 23, 2021

Adopting MVVM

Following the MVVM design pattern in Android development is now easier with DataBinding and Hilt!