New Permission System

[martinheidegger]

Problem
We need a permission system that allows users to connect multiple devices that they control to one, equal, space. To fix this, following concept:

Permission system based on hypercores

Goal:

• a group of clients.
• the group is identifiable for its duration of its existence.
• each member may be offline at any given time.
• each member can request to add (or remove) other clients safely to the group.
• confirmation is only given when enough clients sign a request.
• operations on the group can be done asynchronously in a way that a request may be started at one member and finished by another member.
• the system can be synchronized both blind (zero-knowledge-peer) and seeing (with decryption key).
• one member can not impersonate other members in the group.

Non-Goal:

• Provide detailed permissions

Vocabulary

• client - a device or identifyable through an unique id, needs to have following secrets, may or may not be known to others.
• group - a set of clients, clearly identifyable through an unique id.
• member - a client that is currently part of the group.
• known-client - a client that is or has-been part of the group.
• shard - a part (or the whole) of the group secret that is known to the client.
• feed - a append-only log (aka. hypercore)
• entry - a single entry of many a feed feed
• request - an entry to the feed that is not-yet signed.
• write - an special kind of entry contains the request and all the known signatures for the request, it is also additionally signed by the member that initiated the write
• confirm-receipt - a special kind of entry that verifies that a member received it's new shard.
• group-sign-key - a secret that allows to write to the group feed. the group-sign-key is turned into shards using Shamirs Secrets
• group-feed - a feed of writes, contains all the writes that were signed, its the same for all clients - the source of truth.
• client-feed - a feed of requests, contains all the requests, one per known-client and it also contains writes and confirm-receipt.
• ephemeral - data that is not persisted but may be communicated between the known peers.
• outbox - a set of ephemeral data that contains shards used to restore a group-sign-key or new shards to be stored by each of the members.
• requestee - the member that initially created the request.
• state - exposed information of the client through the API. The state exposes information about the client-state and request-state
  • client-state - state of a client as seen by another clients permission
    • sole-owner - client-state of a client when the group has only one client. the client can freely change other operations.
    • pending-add - client-state of a client that has been requested to be added to the group but hasn't finally been added.
    • confirming-add - client-state of a client that has been confirmed to be added to the group but hasn't received the good news yet!
    • pending-remove - client-state of a client that was requested to be removed from the group.
    • removed - client-state of a client that was successfully removed.
    • member - client-state of a client that has been added to the group successfully.
  • request-state - state of a request by a member
    • pending - initial state of the request
    • aborted - aborted by the writer prior to writing it to the group-feed
    • processed - request has been added to the group-feed
• sync - event when the latest known information of all clients has been exchanged with each other.

Usage

const { createGroup, Member, Group } = require('@consento/crypto')

const { member: alice } = await createGroup({ storage: ram() })
alice.public // Public Connection object, known to other clients that can be used to read data from alice or send data to alice
alice.private // Private Connection object, known to alice that can be used to read data to alice or send data to the group
alice.groupReader // Reader to read data from the group
alice.toJSON() // { public, private, group } = The entire identity of client
alice.groupState // A runtime-only reduced set of information that contains information about all members and requests, mobx object for real-time observation
alice.replicate() // Opens a replication feed that allows to replicate the group's status (hypercore based)
alice.version // Vector number: length of all `known-client-feeds` together with the length for the `group-feed` 

alice.groupState.sync = { [alice.public.connectionKey]: 1 } // known version for every other member

const bob = new Member({ groupReader: alice.groupReader, /* can be restored using the private, public information of .toJSON() */, storage: ram() })
alice.groupState.member[bob.public.connectionKey] // undefined - not added unknown monster

const requestId = await alice.addSibling(bob.public.connectionKey) // Adds a new request to alice's client-feed and signs immediately the output, because only alice is a sibling.
await alice.removeSibling(bob.public.connectionKey) // Adds a new request to alice's client-feed but 
alice.groupState.member[bob.public.connectionKey] // 'confirming-add' - confirmed by alice but since bob doesn't know about it yet, we don't continue on it
alice.groupState.request[requestId] // 'pending'

await alice.abort(requestId) // Means to abort a previous request, can only be done by the peer that requested this action.

// Synching two clients
const stream = alice.replicate()
stream.pipe(bob.replicate()).pipe(stream)

new Group({ reader: alice.groupReader, storage: ram() }) // equals to alice.groupState - Allows to Replicate a group and read data to it, but not interact with the group.

hypercore(alice.groupReader.verifyKey) // the hypercore with all the "writes" - `group-feed`
hypercore(alice.public.output.verifyKey) // the hypercore for all the requests that alice created (content can not be read without decryptKey) - `client-feed`

const bobToAlice = alice.ephemeral.boxes[concat(bob.public.connectionKey, alice.public.connectionKey)] // Data encrypted for bob to alice 
const aliceToBob = alice.ephemeral.boxes[concat(alice.public.connectionKey, bob.public.connectionKey)] // Data encrypted for alice to bob

bobToAlice.shard
bobToAlice.signatures[requestId] // Signature for a given request
alice.ephemeral.version // Version for the data in the boxes - while a sync is running this may be different than `alice.version`

Security and Data persistence

A client can only append to the group-feed with the group-secret. As long as a member is the only member of group, this secret will be stored by that client, but with the second client added the group-secret will be split into shards, to recreate the group-secret we need a threshold of n=amount(members) - 1 different shards. (exception: with 1 and 2 where we need all shards)

To allow the deletion of a member we need n - 1 shards, else the last member is going to be tricky. If we would allow n - 2 shards it may be the case that the writing member doesn't have the latest version of the group-feed which would cause a data integrity error.

During a sync process, when a member notices a new pending-request, it signs the the request and send it together with shards to each other member. Once a member received enough shards and valid signatures, it restores the group-sign-key and uses it to write the request, with the signatures, to the group-feed.

That member then has to "forget" about the other shards, and - if the members have changed - create a new set of shards for each new siblings and start distributing them as ephemeral data to all other members.

The integrity of the group-feed can be verified by going through the signatures of every request.

If the integrity of a feed-entry can not be verified, the changes will not be accepted as in-sync and the member that wrote the change will be automatically requested to be removed (faulty member). If an entry does not even have a signature, it is assumed that the group-feed-secret has been leaked. The peer sending the new entry will be put on a black-list, the feed will not be further processed and a recommendation is given to the user to create a new group Note: the black-listing makes sure that the integrity stays intact and the system keeps on working, but it is still open to a ddos-attack

A risk factor is if a member or known-members key is lost and may be repurposed for forking the group-feed. To avoid that members need to use stream-ciphers to store data when writing to the client-feeds and clients only store the last vector on their device. If it happens to be compromised they can not create a wrong feed.

Each client needs to always store their own shard, the group-feed and the date of all known-clients. This is done to preserve the history of the operations and make the whole log audit-able.

The ephemeral data needs to also be stored, but only until the member has shared a confirmed-receipt for the arrival.

The sharing of ephemeral data needs to be in steps: Alice sends ephemeral data to Bob; Bob looks if they can do anything with that ephemeral data and only if they can't Bob also shares their ephemeral data with Alice. This is done to prevent parallel writes to the group-feed.

[martinheidegger]

@RangerMauve and I just had an offline call. - I will need to rethink the specification. Rethought.

[martinheidegger]

Updated the proposal for the use of signvectors and signing in feeds.

[dkastl]

The Hyperledger people eventually call client an agent.
Is client something used in hypercores?

[RangerMauve]

agent seems like a good name. 👍

[dkastl]

I think other terms are good and the meaning of group and member here is probably exactly what we imagine it should be.

[RangerMauve]

Writing notes about this https://hackmd.io/6ApfbzeSRC6Hw1nnez6vLQ?edit

[cblgh]

perhaps also of interest @martinheidegger (if anything as related work):

• ssb's new private groups spec, which makes use of cryptographically private envelopes—it is attempting to create private groups on ssb, similar to how signal's groups work
• my trustnet, (code) thesis project, which allows for individuals to create a subjective and trusted subset of people from a larger set of individuals. it could easily be made of use to only perform actions if x of the y people within yr trusted group issue agreeance
• dark crystal by magma collective, which has implemented shamir's secret sharing for threshold encryption

[martinheidegger]

Note: to follow the current progress on the permission system: consento-org/group#1