Skip to content

Commit

Permalink
README (bluesky-social#69)
Browse files Browse the repository at this point in the history 
* scrips from root

* readme work

* readme updates

* more readme

* change common name
  • Loading branch information
@dholms
dholms authored Apr 10, 2022
1 parent ed2ea87 commit 16ea3bd
Show file tree
Hide file tree
Showing 4 changed files with 112 additions and 191 deletions.
264 changes: 107 additions & 157 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,216 +1,166 @@
# Bluesky hack

This is a proof of concept for a 'decentralized social network'.
Welcome to the Bluesky network prototype 🌞

The demo revolves around two main concepts:
To learn more about the Bluesky network check out our docs on [Network architecture]() and the [Blogpost on self-authenticating data structures]().

**Content Addressing:** Data is stored and distributed in a content addressed manner. A user's profile & post history can be represented by a single hash.
We encourage you to play around with the tools here, but please do not use anything here in production. This is a _proof of concept_ meant to demonstrate the core functionality of the bluesky network.

**User-controlled Keys & Auth:** Users register a public key to their account and sign updates & authorization tokens in the form of [ucans](https://github.com/ucan-wg/ts-ucan/)
## What's here?

## Running the Demo
_Requires Node>=15, and yarn_

There are four components to the demo:
- a "bluesky" server (federated data & identity management)
- the frontend (simple micro-blogging application)
- the command-line (simple micro-blogging CLI)
- a third-party server that a user can delegate permission to to post on it's behalf (think [Buffer](https://buffer.com/))
This is a monorepo containing three packages:

This project is setup in a yarn workspace. To install dependencies for all sub-projects, just run `yarn` from the project root. To build all sub-projects, run `yarn build` from the project root.
- `common`: This is the bluesky SDK that contains implementations of:
- the repository data structure structure
- a sample network namespace (microblogging) with both a full client and delegator client implementation
- an authorization library for working with bluesky-capable UCANs
- some helpers for making calls to a bluesky data server

### Bluesky Server
This package is published on npm as [bsky](https://www.npmjs.com/package/bsky)

Run the main bluesky server

```sh
cd server
yarn dev
```
- `server`: This is an implementation of a bluesky server. For simplicity's sake, it actually combines the function of three "roles" in the network:
- **Identity:**
- maintains a mapping of username -> DID
- **Data:**
- maintains a pointer to the latest root CID of a data repository.
- verifies the authority of pushes to your repo and updates the root.
- servers the repository for pulls.
- acts as a delgatee and makes udpates to your repository for properly authorized requests
- sends updates to other data/indexing servers that have subscribed to a particular DID that it is hosting
- **Indexing:**
- stores an indexed version of repositories that it is hosting or that its user's are following
- returned global view of data including follower lists, aggregated like counts, and user timelines.

Server will be running at `http://localhost:2583`
- `cli`: This is a basic command line interface for interactions with the bluesky network :
- creating a local repository
- registering a user
- creating/editing/deleting posts
- creating/deleting likes
- viewing a user's timeline
- viewing a user's feed

### Frontend

In another console tab, run the frontend

```sh
cd frontend
yarn dev
```
_Note: the cli uses a delegator client at the moment. We are adding the option to use a full client soon._

Go to `http://localhost:3005` to try the demo.

### Command-line

In another console tab, run the CLI

```sh
cd cli
# build, if needed, with `yarn build`
yarn cli
```
## Quick use
_Requires Node>=15, and yarn_

### Third-party server (optional)
To enable third-party posting, run the third-party server as well in another console
```sh
cd third-party
yarn dev
```
Want to jump right in? Follow these steps to get a sample two server network up and running.

Server will be running on `http://localhost:2584`
This demo takes four terminal windows:
- two servers to show of data federation
- two cli clients representing two users on separate servers interaction

## What's going on here?
### Building blocks
- [Ucans](https://github.com/ucan-wg/ts-ucan/): Ucans are distributed user-controlled & signed authorization tokens. They are signed JWTs that indicate what a given user (or keypair) is capable of. A user registers a "root DID" that has full account access, and using that keypair, can delegate some subset of their capabilities (such as posting) to another device, user, or third-party platform.
- [CAR files](https://github.com/ipld/js-car): CAR files are "Content Addressable aRchives". They encode and serialize some set of content addressable objects and allow you to indicate a "root" of the content addressed structure. CAR files allow us to take advantage of content addressing while avoiding the performance hit of content discovery in an in-browser DHT.
- [Hash Array Mapped Trie(HAMT)](https://github.com/rvagg/js-ipld-hashmap): A data structure that functions like a hashmap, but under the hood stores values in a trie.
The number in parantheses tells you which terminal to run each command in

### Registration
The user creates an keypair and signs an empty (no attenuation) UCAN. This servers as proof of ownership of the key.
From project root:
```bash
# install dependencies
(1) yarn

They then send the signed ucan and their requested username to the server.
# build projects
(1) yarn build

The server parses and validates the UCAN, checks to make sure the username is available, and registers the DID that issued the UCAN to the requested username.
# run server
(1) yarn server # runs on localhost:2583

_Note: currently keys are stored in localStorage, which is not a safe location in production_
# in a separate terminal, run a second server
(2) yarn server:alt # runs on localhost:2584

### Posting
The user maintains a merkelized "User Store" that wraps around a HAMT (described in more detail later). When they create a new post, they add it to the HAMT, indexed by the current post count.
# set an env var to store alice's repo in a scoped dir
(3) export SKY_REPO_PATH="~/.sky-alice"

The user signs a UCAN with a valid `POST` permission for their username.
# set an env var to store bob's repo in a scoped dir
(4) export SKY_REPO_PATH="~/.sky-bob"

The user serializes their user store to a CAR file, and sends it to the server, adding the encoded UCAN to the request in the form of a Bearer token.
# register alice
(3) yarn cli init
# prompt with 'alice' for username, 'localhost:2583' for host & true for registration

The server validates the UCAN, ensuring that it has valid `POST` permission, decodes the CAR file, stores it in it's datastore (currently: memory), and updates the user's data root to the root of the CAR file.
# register bob
(4) yarn cli init
# prompt with 'bob' for username, 'localhost:2584' for host & true for registration

### Listing posts
Posts are all public for the time being, so no authentication is needed.
# make a couple posts as alice
(3) yarn cli post "hello world"
(3) yarn cli post "howdy"

The user requests the data for a given DID.
# follow alice as bob
(4) yarn cli follow alice@localhost:2583

The server serializes the user store to a CAR file and sends it to the user.
# like alice's post
(4) yarn cli like alice@localhost:2583 {post_id from alice post} # the post id has the format `3iwc-gvs-ehpk-2s`

The user decodes the CAR file and displays the posts to the user.
# view your timeline
(4) yarn cli timeline

# list your follows
(3/4) yarn cli list follows

## User Store
The user store is a content-addressed merkelized data structure that encodes some basic information about a user and their post history.
# list your followers
(3/4) yarn cli list followers

Posts are stored in a HAMT, indexed by the current post count.
# list your feed
(3/4) yarn cli feed

An [dag-cbor](https://github.com/ipld/js-dag-cbor) encoded [IPLD](https://github.com/ipld/ipld) block contains some basic information about a user and a pointer to the current root of the HAMT.
# list your followers
(3/4) yarn cli list followers

```ts
// dag-cbor encoded user root
type User = {
name: string
did: string
nextPost: number
postsRoot: CID
follows: Follow[]
}
# list your feed
(3/4) yarn cli feed

// stored in HAMT
type Post = {
user: string
text: string
}
# view alice's feed as bob
(4) yarn cli feed alice@localhost:2583

type Follow = {
username: string
did: string
}
# Keep playing around. Try unliking, deleting or editing posts, or add a third user into the mix! They can be registered to one of the existing servers
```

The CID of the IPLD-encoded "User" block is then signed by the user and added to to an "Commit" IPLD object:
```ts
type Commit = {
user: CID
sig: Uint8Array
}
```
## Documentation
We are putting together more detailed documentation for the server API as well as the SDK.

The CID of the Commit serves as the root of the datastructure.
If you are inclined to play with either, your best option is to check the tests to see how to use each part of the library.

## Persistence
User data is persisted in [Level](https://github.com/Level/level) which is a simple key-value store. This **Blockstore** is a mapping of `CID -> Bytes` and represents the consituent blocks that make up the UserStore.
Specifically:

## Server API
The server uses [Ucans](https://github.com/ucan-wg/ts-ucan/) for authorization.
`server/test/delegator.ts` contains an example of a delegator client (the simplest client to work with)

### `POST /register`
`server/test/indexer.ts` contains an example of multiple users interaction on a 2 server network

**Body:**
```
{
name: string
}
```
`common/test/repo.ts` contains an examples of directly performing updates to a user repo

**Auth:**
A Ucan no attenuation. This token is just used to prove key ownership.
`common/test/microblog.ts` contains an examples of using the microblog library to create/update microblog namespace objects in the repository

---
For communicating directly with the server api, there is a schema above each route that details the exact parameters it expects to receive. Any post route will require a valid UCAN as a Bearer token. We recommend using the SDK make these requests as these tokens can be difficult to roll by hand.

### `POST /update`
## Notes for code spelunkers
We hope you jump into the code to explore these concepts alongside us! Feel free to create issues/discussion with problems that you come across or questions that you have.

**Body:**
Binary [CAR file](https://github.com/ipld/go-car) representing a valid user store
A few notes for the curious ones that find themselves trawling the depths of the code:

**Auth:**
A valid Ucan with attenuation for the following resource:
```
{
'bluesky': '${USERNAME}'
'cap': 'POST'
}
```
### Namespaces

### `GET /users`
Data is separated in the user repository by namespace.

**Params:**
- `id`: User's DID
A user's microblogging data live separately from their community forum data which lives separate from their long form writing data and so on. Each namespace following its respective data spec.

**Returns:**
An array of all user DIDs
```
[
did:key:abcdef.....,
did:key:123456....,
...
]
```
We've implemented only one sample namespace here: microblogging.

### `GET /user/:id`
You'll notice that we sometimes switch between using the words "interactions" and "likes". The reason for this is that the user repository speaks in terms of "interactions", which is any sort of interaction generated by an application. These may be likes, retweets, upvotes, shares, rsvps, etc. Our current (simplified) microblogging spec only allows for "likes" as interactions.

**Params:**
- `id`: User's DID
Therefore we try to talk about the general concept as "interactions" and the particular as "likes". There is some gray area here where those two concepts blur together and will be better demarcated as we develop our schema system.

**Returns:**
Binary [CAR file](https://github.com/ipld/go-car) representing the user's current user store
### DIDs and UCANs

### `Get /.well-known/did.json`
In this prototype a user's root DID is a simple `did:key`. In the future, these will be more permanent identifiers such as `did:bsky` (read our proposal in the architecture docs) or `did:ion`.

**Returns:**
The server's DID
```
{
id: "did:key:z6Mkmi4eUvWtRAP6PNB7MnGfUFdLkGe255ftW9sGo28uv44g"
}
```
You'll notice that we delegate a UCAN from the root key to the root key (which is a noop), this is to mirror the process of receiving a fully delegated UCAN _from your actual root key_ to a _fully permissioned device key_.

### `Get /.well-known/webfinger?resource=${username}`
You'll also notice that the DID for the microblogging namespace is just `did:bsky:microblog` (which is not an actual valid DID). This is a stand in until we have an addressed network for schemas.

**Params:**
- `resource`: The user's name
UCAN permissions are also simplified at the current moment, allowing for scoped `WRITE` permission or full-repo `MAINTENANCE` permission. These permissions will be expanding in the future to allow presenting CRUD operations, and more detailed maintenance (ie creation vs merging vs cleanup, etc)

**Returns:**
A (very sparse) webfinger document. It currently only contains the user's DID.
```
{
id: "did:key:zAbc...."
}
```
### Client types

In the architecture overview, we specify three client types: full, light, and delegator. This library only contains implementaions of full and delegator. Thus we use delegator for light weight operations and a full client when we want the entire repository.

The main ramification of this is that data server subscribers must receive the _full repo_ of the users that they subscribe to. Once we add in light clients, they can receive only the _sections_ the repo that they are interested in (for instance a single post or a like) while having the same trust model as a full repo.

34 changes: 1 addition & 33 deletions cli/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,35 +1,3 @@
# @bluesky/cli

This is the commandline interface to interact with the SKY protocol.

## Quick use guide
Run `yarn` from the project root to install all dependencies
Run `yarn build` within the `common/` folder to build the core bsky package
Run `yarn build` within this `cli/` folder to build the cli
Use it with `yarn cli {cmd}` from inside the `cli/` folder

To get this working, you'll need a server running. Pop over to `server/` and run `yarn dev`

We're running this demo with two users on our system.

Therefore, open two terminals:
in the first terminal, run `export SKY_REPO_PATH="~/.sky-alice"`
in the second terminal, run `export SKY_REPO_PATH="~/.sky-bob"`

In each terminal run
`yarn cli init` & prompt them with different usernames, for instance 'alice' & 'bob'
when prompted for a server, use the address of your dev server (`http://localhost:2583` by default)
when prompted if you want to register, use the default of `true`

- make a post as alice with `yarn cli post "hello world"`
- follow alice as bob with `yarn cli follow alice`
- like alice's post with `yarn cli like alice {post_id from alice's post}`
- view your timeline with `yarn cli timeline`
- unlike alice's post with `yarn cli unlike alice {post_id from alice's post}`
- list your follows with `yarn cli list follows`
- list your followers with `yarn cli list followers`
- view your feed with `yarn cli feed`
- view someone else's feed with `yarn cli feed alice`



The command line interface for the SKY protocol.
2 changes: 1 addition & 1 deletion common/package.json
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@bluesky/common",
"version": "1.0.0",
"version": "0.0.1",
"main": "dist/index.js",
"type": "module",
"scripts": {
Expand Down
3 changes: 3 additions & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
],
"scripts": {
"build": "wsrun -c \"build\"",
"server": "yarn workspace @bluesky/server start",
"server:alt": "ENV=alt yarn server",
"cli": "yarn workspace @bluesky/cli cli",
"wipe-db": "yarn workspace @bluesky/server wipe-db"
},
"devDependencies": {
Expand Down

0 comments on commit 16ea3bd

Please sign in to comment.