Skip to content

Commit

Permalink
Added multithreaded support
Browse files Browse the repository at this point in the history 
It's now possible to fire up multiple event loops in separate
goroutines. All that's needed is to set the `events.NumLoops` options
before calling `Serve`.

There are a few breaking API changes.

- The events pass an evio.Conn param that represents the unique
incoming socket connection.
- Prewrite and Postwrite events have been removed.
- Wake and Dial functions have been removed.
- The Transform utility has been removed.

The older version has been tagged as `v0.1.0` for vendoring purposes.
  • Loading branch information
@tidwall
tidwall committed May 23, 2018
1 parent 751f591 commit dd88755
Show file tree
Hide file tree
Showing 21 changed files with 1,529 additions and 2,641 deletions.
146 changes: 22 additions & 124 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,23 +17,20 @@ This project is not intended to be a general purpose replacement for the standar

You would not want to use this framework if you need to handle long-running requests (milliseconds or more). For example, a web api that needs to connect to a mongo database, authenticate, and respond; just use the Go net/http package instead.

There are many popular event loop based applications in the wild such as Nginx, Haproxy, Redis, and Memcached. All of these are single-threaded and very fast and written in C.

The reason I wrote this framework is so I can build certain network services that perform like the C apps above, but I also want to continue to work in Go.

There are many popular event loop based applications in the wild such as Nginx, Haproxy, Redis, and Memcached. All of these are very fast and written in C.

The reason I wrote this framework is so that I can build certain networking services that perform like the C apps above, but I also want to continue to work in Go.

## Features

- [Fast](#performance) single-threaded event loop
- [Fast](#performance) single-threaded or [multithreaded](#multithreaded) event loop
- Built-in [load balancing](#load-balancing) options
- Simple API
- Low memory usage
- Supports tcp, [udp](#udp), and unix sockets
- Allows [multiple network binding](#multiple-addresses) on the same event loop
- Flexible [ticker](#ticker) event
- Fallback for non-epoll/kqueue operating systems by simulating events with the [net](https://golang.org/pkg/net/) package
- Ability to [wake up](#wake-up) connections from long running background operations
- [Dial](#dial-out) an outbound connection and process/proxy on the event loop
- [SO_REUSEPORT](#so_reuseport) socket option

## Getting Started
Expand Down Expand Up @@ -61,7 +58,7 @@ import "github.com/tidwall/evio"

func main() {
var events evio.Events
events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
events.Data = func(c evio.Conn, in []byte) (out []byte, action evio.Action) {
out = in
return
}
Expand Down Expand Up @@ -89,8 +86,6 @@ The event type has a bunch of handy events:
- `Closed` fires when a connection has closed.
- `Detach` fires when a connection has been detached using the `Detach` return action.
- `Data` fires when the server receives new data from a connection.
- `Prewrite` fires prior to all write attempts from the server.
- `Postwrite` fires immediately after every write attempt.
- `Tick` fires immediately after the server starts and will fire again after a specified interval.

### Multiple addresses
Expand All @@ -114,127 +109,30 @@ events.Tick = func() (delay time.Duration, action Action){
}
```

### Wake up

A connection can be woken up using the `Wake` function that is made available through the `Serving` event. This is useful for when you need to offload an operation to a background goroutine and then later notify the event loop that it's time to send some data.

Example echo server that when encountering the line "exec" it waits 5 seconds before responding.

```go
var srv evio.Server
var mu sync.Mutex
var execs = make(map[int]int)

events.Serving = func(srvin evio.Server) (action evio.Action) {
srv = srvin // hang on to the server control, which has the Wake function
return
}
events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
if in == nil {
// look for `in` param equal to `nil` following a wake call.
mu.Lock()
for execs[id] > 0 {
out = append(out, "exec\r\n"...)
execs[id]--
}
mu.Unlock()
} else if string(in) == "exec\r\n" {
go func(){
// do some long running operation
time.Sleep(time.Second*5)
mu.Lock()
execs[id]++
mu.Unlock()
srv.Wake(id)
}()
} else {
out = in
}
return
}
```

### Dial out

An outbound connection can be created by using the `Dial` function that is made available through the `Serving` event. Dialing a new connection will return a new connection ID and attach that connection to the event loop in the same manner as incoming connections. This operation is completely non-blocking including any DNS resolution.

All new outbound connection attempts will immediately fire an `Opened` event and end with a `Closed` event. A failed connection will send the connection error through the `Closed` event.

```go
var srv evio.Server
var mu sync.Mutex
var execs = make(map[int]int)

events.Serving = func(srvin evio.Server) (action evio.Action) {
srv = srvin // hang on to the server control, which has the Dial function
return
}
events.Data = func(id int, in []byte) (out []byte, action evio.Action) {
if string(in) == "dial\r\n" {
id := srv.Dial("tcp://google.com:80")
// We now established an outbound connection to google.
// Treat it like you would incoming connection.
} else {
out = in
}
return
}
```

### Data translations

The `Translate` function wraps events and provides a `ReadWriter` that can be used to translate data off the wire from one format to another. This can be useful for transparently adding compression or encryption.

For example, let's say we need TLS support:

```go
var events Events

// ... fill the events with happy functions

cer, err := tls.LoadX509KeyPair("certs/ssl-cert-snakeoil.pem", "certs/ssl-cert-snakeoil.key")
if err != nil {
log.Fatal(err)
}
config := &tls.Config{Certificates: []tls.Certificate{cer}}

// wrap the events with a TLS translator

events = evio.Translate(events, nil,
func(id int, rw io.ReadWriter) io.ReadWriter {
return tls.Server(evio.NopConn(rw), config)
},
)

log.Fatal(evio.Serve(events, "tcp://0.0.0.0:443"))
```
## UDP

Here we wrapped the event with a TLS translator. The `evio.NopConn` function is used to converts the `ReadWriter` a `net.Conn` so the `tls.Server()` call will work.
The `Serve` function can bind to UDP addresses.

There's a working TLS example at [examples/http-server/main.go](examples/http-server/main.go) that binds to port 8080 and 4443 using an developer SSL certificate. The 8080 connections will be insecure and the 4443 will be secure.
- All incoming and outgoing packets are not buffered and sent individually.
- The `Opened` and `Closed` events are not availble for UDP sockets, only the `Data` event.

```sh
$ cd examples/http-server
$ go run main.go --tlscert example.pem
2017/11/02 06:24:33 http server started on port 8080
2017/11/02 06:24:33 https server started on port 4443
```
## Multithreaded

```sh
$ curl http://localhost:8080
Hello World!
$ curl -k https://localhost:4443
Hello World!
```
The `events.NumLoops` options sets the number of loops to use for the server.
Setting this to a value greater than 1 will effectively make the server multithreaded for multi-core machines.
Which means you must take care with synchonizing memory between all event callbacks.
Setting to 0 or 1 will run the server single-threaded.
Setting to -1 will automatically assign this value equal to `runtime.NumProcs()`.

## UDP
## Load balancing

The `Serve` function can bind to UDP addresses.
The `events.LoadBalance` options sets the load balancing method.
Load balancing is always a best effort to attempt to distribute the incoming connections between multiple loops.
This option is only available when `events.NumLoops` is set.

- The `Opened` event will fire when a UDP packet is received from a new remote address.
- The `Closed` event will fire when the server is shutdown or the `Close` action is explicitly returned from an event.
- The `Wake` and `Dial` operations are not available to UDP connections.
- All incoming and outgoing packets are not buffered and sent individually.
- `Random` requests that connections are randomly distributed.
- `RoundRobin` requests that connections are distributed to a loop in a round-robin fashion.
- `LeastConnections` assigns the next accepted connection to the loop with the least number of active connections.

## SO_REUSEPORT

Expand Down
Loading

0 comments on commit dd88755

Please sign in to comment.