A Guide to Developing TrebleShot Based Applications

Veli Tasalı yazdı   ·   09 July 2019

design architecture

TrebleShot

TrebleShot

Attention! This document is obsolete, and you should avoid studying it. 2.0 superseded this specification.


TrebleShot aims to improve the basic file exchanging methods by eliminating the issues that the existing apps fail to address.

It doesn't just push bytes to the other side. It tries to collect information about the environment so that the user can benefit from it later.

This guide illustrates how it does that and briefly explains its communication protocol.

First Step

This guide doesn't include any code. If you need to see a real implementation, you have two resources: TrebleShot for Android and Desktop.

What is the Communication Server?

The Communication Server handles the requests and is a way to broadcast the identity of the device.

Which port does it use?

It runs on the port 1128. That port number is static and never changes so that connections can still be possible when services like mDNS are not available.

What type of connection?

The Communication Server is an implementation of CoolSocket, a TCP socket implementation in itself.

A Basic CoolSocket Wrapper

This section covers how a CoolSocket server and client should behave. Skip to the next section if you are going to use an existing implementation.

You can use CoolSocket Client for Android for testing.

A single packet

You can easily replicate the behavior of CoolSocket if your platform is not supported. A CoolSocket packet consists of two parts: header & data.

The header should be in JSON format containing the required field length which represents the length of the data in a long integer format.

The field length doesn't necessarily mean the length of the data in text format. You should treat it as a byte. The preferred encoding should be UTF-8 if it is in plain text.

Between the header and the data, HEADER_END is expected with line breaks (\n) at the start and end. This will mark the beginning of data the and the end of the header.

The contents of the data can be anything. It doesn't have to be in plain text.

A full packet for

Hi!

will look like:

{"length": 3}
HEADER_END
Hi!

Preserving the Keep-Alive State

CoolSocket connections are keep-alive. They should be treated as such unless you are aiming at singe-direction one time read or write connections.

  1. You should not close the connections after you read a packet or sent one. Instead, you should do nothing.

  2. You should read when the other side is sending or do otherwise if the other way around is the case.

  3. Due to timing issues, you should not repeat the same call; i.e., reading 2 times. The reason is when you do the same operation twice or more, the previous operation may steal the input/output from the next operation. This would cause the data to look corrupted. If you need to repeat operations, then, send arbitrary data while the other side is aware and do it afterward.

The Structure

This section talks about the structure of the servers. There is no code involved and everything is a representation of the actual protocol. Below you will see a scheme which will tell you what is what.

Reading the Diagrams

The communication data is in JSON format. The following diagram is a text representation of that. For instance, when you see this:

  • data_key
    • key_first = String that defines the first key
    • other
      • key_other = Integer that represents the maximum byte allowed

The actual data will be this:

{
    "data_key": {
        "key_first": "FirstKey",
        "other": {
            "key_other": 8096
        }
    }
}

Keywords and Their Meanings

  • PutReply Add the values below it into the reply.
  • Receive Receive the response of the other side.
  • Return Stop executing. This process is finished.
  • SendReply Send the reply you PutReply into and clear the reply index.
  • See Refer to the section with the same title.
  • break Break out of the referred scope. For instance the loop scope.
  • Interrupt Interrupt the specified task and flag it.
  • loop Does this until it finds a break to go out of the loop.
  • When A condition in text format
  • Else The opposite of the same level equation.
  • = (An equals condition)
  • != (A not equals condition)
  • true Positive boolean (programming language specific).
  • false Negative boolean (programming language specific).
  • undefined Undefined in a map (programming language-specific).

A Suggestion on Using Constants

You should turn the keywords into constants (e.g. REQUEST to Keyword.REQUEST) to avoid misspelling.

The Communication Server

It consists of two related parts.

  1. handshake - clients send information about the device info and the version info, etc.

  2. post-handshake - when clients say there is more coming.

The data-flow of the communication server is shown below;

  • Receive the first response
  • PutReply the device info of your device and version info of the client
    • deviceInfo
      • deviceId = The string that is unique to your device, usually the serial number of it
      • brand = The manufacturer of your device
      • model = The model of your device
      • user = The nickname you chose for your device
    • appInfo
      • versionCode = The version code of the client identical to the official versions' (e.g. 91)
      • versionName = The version name of the client in the string format (e.g. 2.0)
  • handshakeRequired = true
    • Part: handshake
    • SendReply
      • The data containing your device information should be sent in this scope.
    • handshakeOnly = true
      • This request is hanshake only, which means the other side only wants to know who you are.
      • Receive receive the obsolete data (for compatibility reasons).
      • Return
        • The request is fulfilled.
    • SendReply: Send the device information with the first reply if hanshakeRequired = true. If false, it should be sent with the second reply. You don't need to send the same reply twice (unless that is what you are aiming at).
    • Receive the second response
  • Part: post-handshake
  • You should check if the device is known and can be trusted. You can connect to its communication server to gather information.
  • request = requestTransfer
    • The client requests a file transfer operation. Consider doing this asynchronously and returning the result immediately.
    • groupId = The long integer representing this transfer group. The long integer should be treated as a string and later as a long integer if your JSON parser doesn't allow long integers.
    • filesIndex = Array of JSON objects containing files
      • requestId = The long integer representing the file
      • file = The file name
      • fileMime = The MIME-Type of the file
      • fileSize = The file size in byte format as a long integer
      • directory = The directory that the file should be put (may be null)
  • request = requestResponse
    • The other side will make this request to inform you about your previous requestTransfer request. You can use this to remove the operation from the database when the other side rejects your transfer request.
    • isAccepted = true if approved.
    • groupId = The long integer for the specific transfer
  • request = requestClipboard
    • This is a text-stream request.
    • clipboardText = The text
  • request = requestAcquaintance
    • The client wants you to perform a task following the user's location on the client; e.g., the user is in Receive section of the client in which case you can open a window that shows the client's info.
  • request = requestHandshake
    • The client wants to know if you are going to approve a request it is going to make. You should return result = true if you will.
  • request = requestStartTransfer
    • The client wants you to start the transfer.
    • groupId = The long integer that represents the group of a transfer that should be started
    • PutReply
      • result = true if you will start the process
      • error optional = notAllowed or notFound or errorRequireTrustZone or errorUnknown or notAccessible
        • notAllowed = The client is blocked or not allowed
        • notFound = The input given by the client is not matching what you have
        • errorRequireTrustZone = The operation is only permitted for TrustZone devices. TrustZone is a way of defining a secure zone for select devices
        • errorUnknown = The error is not identifiable (or the developer is lazy)
        • notAccessible = The data that the client is trying access doesn't belong to it
  • SendReply
    • result = true when the requested operation was successful or false if it was not.

The Seamless Server

This is the server that handles the sending of the files. It pushes files to the clients we requested file transfer operations from. This server runs on the port 58762 (or 5TREB).

The workflow:

  • Receive the transfer information
  • groupId = The long integer that represents the transfer group
  • deviceId = (added in build 91) The string of the device serial
  • PutReply
    • result = true if the given groupId is known and deviceId has permission to receive.
    • SendReply
  • result = false
    • Return
  • loop Is connection open?
    • Receive the transfer info
    • result = undefined or result = false
      • jobDone != undefined and jobDone = true
        • The task is completed.
      • Else
        • Interrupt The task failed for some reason, and we were not notified by the receiver.
    • Else result = true
      • Transfer happens here.
      • requestId = The long integer representing the file that should be sent
      • socketPort = The TCP socket port that you connect to write the bytes of the file
      • skippedBytes = The previous position of the byte position of file
      • When everything is okay
        • PutReply:
          • result = true
          • sizeChanged optional = The size of the file has changed to
        • SendReply
      • Else the file cannot be sent
        • When the file cannot be read
          • PutReply
            • result = false
            • error = notAccessible
            • flagGroupExists* = true When*** the group exists
        • Else requestId = invalid
          • PutReply
            • result = false
            • error = notFound
            • flagGroupExists* = true When*** the group exists
        • Else the task is somehow cancelled, and did not complete
          • PutReply
            • result = false
            • jobDone = false
        • SendReply

The Seamless Client

The seamless client communicates with the seamless server when receiving files. This is the opposite side of the seamless server and the port you are going to connect is 58762 (or 5TREB).

The workflow is:

  • The first request is made to the communication server.
  • PutReply
    • handshakeRequired = true
    • handshakeOnly = false
    • deviceId = The unique string that represents your device
    • secureKey optional = The unique integer that the other side gave you to use establish secure connections
  • SendReply
  • Receive

The received data will be similar to this:

{
    "deviceInfo": {
        "deviceId": "h5k34h5kj34h5",
        "brand": "Samsung",
        "model": "I8852",
        "user": "Veli's Device"
    },
    "appInfo": {
        "versionCode": 98,
        "versionName": "1.4.3"
    },
    "result": true
}
  • PutReply
    • Ask the target if we can start a file transfer process.
    • request = requestHandshake
  • SendReply
  • Receive
    • result = false
      • The sender rejected the request.
      • Return
  • Connect to the seamless server of the sender to receive the files.
  • PutReply
    • deviceId = Your device ID in the string format
    • groupId = The long integer that represents the transfer group
  • SendReply
  • Receive
    • result = true
      • loop the connection is open
        • Gather the first available transfer you need to receive for this transfer group (which you save somewhere).
        • When there is not any
          • This usually means the task is done or all the files have been flagged with errors.
          • break
        • PutReply
          • requestId = The long integer that represents the file you expect to receive
          • groupId = The long integer that represents the transfer group
          • socketPort = The TCP socket port that you open for the sender to transfer the file
          • result = true
          • When there is already a file in the temporary location of the file you will receive
            • Notify the other side about it. This file may be a leftover from the same transfer group and you can append the upcoming data onto it. Make sure the size of that leftover is not larger than the file you will receive. If so, make you are using unique paths for temporary files because that scenario should be unlikely.
            • skippedBytes = The long integer that represents the size of the leftover file
        • SendReply
          • result = false
            • jobDone != undefined and jobDone = false
              • Interrupt
            • Else flag != undefined and flag = flagGroupExists
              • The group exists, but the file has issues. Use the error field to determine the error type.
              • Flag this file so that you will not see it in the next iteration.
            • Else
              • You can receive the file here.
              • sizeChanged != undefined
                • The file has a new length (changed during after it was reported). This should not be a problem when the file doesn't have leftover data larger than 0 (zero) on its temporary location. You can update the size field of the file with the sizeChanged field.
                • When the file has leftover data, you should not append the file. Instead, change the temporary location for now.
              • Receive the file by reading from the server socket you created for it.
      • You are out of the loop.
      • PutReply
        • Tell the other side that you will exit.
        • result = false You are about to go out of the transfer scope.
        • jobDone = true if the transfer did not interrupt and you don't have any files left to receive, or false if otherwise
    • Else
      • The groupId does not correspond to a valid transfer group on the sender side. Use the error field to detect errors.
Önceki Gönderi
This Summer
Sonraki Gönderi
Switching to ArchLinux

Dizi Understanding the Architecture of TrebleShot

Bu gönderi bir dizinin parçasıdır. Bu bölüm aynı dizideki diğer gönderileri gösterir.

TrebleShot Viki

A scene from Where the Wild Things Are (2009) where Carol and Max stand next to each other
en
A Standalone Approach to Threads

Veli Tasalı yazdı ·  25 February 2019