Skip to content

Commit

Permalink
Added support for tracing to the api-server service.
Browse files Browse the repository at this point in the history
- Added a tracing backend service powered by Jaeger.
- Instrumented api-server service using the jaeger-client Opentracing library.
  • Loading branch information
Uzziah Eyee committed May 4, 2019
1 parent fc7f7a7 commit cb6a189
Show file tree
Hide file tree
Showing 5 changed files with 224 additions and 14 deletions.
38 changes: 32 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ A test project for learning microservices observability through: **centralized l
1. Clone this repo and open the root folder in an IDE like *Visual Studio Code*.

2. For each microservice, rename `example.env` to `.env` and supply the needed secrets.
> TODO: Is there a way to eliminate this friction?
3. Start all microservices in *development mode*.

Expand All @@ -26,6 +27,10 @@ A test project for learning microservices observability through: **centralized l
4. Attach the IDE's debugger to the desired service. Follow these steps in Visual Studio Code: *shift+cmd+D > Select a debug configuration > F5*.
> All Visual Studio Code debug configurations are stored in *.vscode/launch.json*. You can modify configs as you see fit.
5. To view tracing information, access the Jaeger UI on http://localhost:16686

> In development, the tracing backend is a single service (*tracing-backend*) for simplicity, and traces are stored in-memory. However, in production, the tracing backend will setup as multiple services (running on multiple containers), and traces will be persisted in an external store like Elasticsearch.
### Useful dev commands

# list all running services
Expand All @@ -39,7 +44,8 @@ A test project for learning microservices observability through: **centralized l
up -d --no-deps --build [service-name]

# tail logs from all [or specific] service
docker-compose logs -f [service-name]
docker-compose -f docker-compose.dev.yml \
logs -f [service-name]

## Production Setup

Expand All @@ -60,13 +66,13 @@ I wrote an accompanying [article](https://hackernoon.com/monitoring-containerize

- You can update a single service/container while the entire suite is running using the following command

docker-compose -f docker-compose.dev.yml \
up -d --no-deps --build <service_name>
docker-compose -f docker-compose.dev.yml \
up -d --no-deps --build <service_name>

docker-compose -f docker-compose.prod.yml \
up -d --no-deps --build <service_name>
docker-compose -f docker-compose.prod.yml \
up -d --no-deps --build <service_name>

> `--build` recreates the container from its image/dockerfile and `--no-deps` ensures dependent services aren't affected.
> `--build` recreates the container from its image/dockerfile and `--no-deps` ensures dependent services aren't affected.
### Docker Networking

Expand All @@ -89,6 +95,26 @@ By default each containerized process runs in an isolated network namespace. For

3. Use sub-second precision for fluentd timestamps (probably best to use nanoseconds.)

### Environment variables used by Jaeger components

1. [jaeger-client-node](https://github.com/jaegertracing/jaeger-client-node#environment-variables)

JAEGER_SERVICE_NAME
JAEGER_SAMPLER_TYPE
JAEGER_SAMPLER_PARAM
JAEGER_AGENT_HOST
JAEGER_AGENT_PORT
JAEGER_ENDPOINT

2. Jaeger Agent
TODO

3. Jaeger Collector (and a Storage Backend)
TODO

4. Jaeger Query Service and UI
TODO

## Some References

https://medium.com/lucjuggery/docker-in-development-with-nodemon-d500366e74df
Expand Down
57 changes: 50 additions & 7 deletions api-server/src/app.js
Original file line number Diff line number Diff line change
@@ -1,17 +1,60 @@
const app = require('express')()
const uuid = require('uuid/v1')
const axios = require('axios')

// For portability, we initialize tracer from envars instead of local options.
// See: https://www.npmjs.com/package/jaeger-client#environment-variables
var initTracer = require('jaeger-client').initTracerFromEnv;
var tracer = initTracer();

app.use('/api/v1/tokens', (req, res) => {
const span = tracer.startSpan('token-request')

console.log(`Handling request for token`)
res.json({token: uuid()})

span.finish()
})
app.use('/api/v1/whereami', (req, res) => {
// TODO: get geolocation info from IP address
// TODO: get weather info from geolocation
// TODO: return everything
let data = {'info': null}
res.json(data)

app.use('/api/v1/whereami', async (req, res, next) => {
const parentSpan = tracer.startSpan('whereami-request')

try {
// get location of IP Address
const IP = '23.16.76.104'
const locSpan = tracer.startSpan('get-location', {childOf: parentSpan})
const location = await axios.get(`http://ip-api.com/json/${IP}`)
locSpan.finish()
const {lat, lon, city, country} = location.data

// do some other async task
const fakeSpan = tracer.startSpan('fake-fetch', {childOf: parentSpan})
const _ = await fakeFetch(100, 0.7)
fakeSpan.finish()

// return results
const data = {lat: lat, lon: lon, city: city, country: country}
res.json(data)
} catch(err) {
parentSpan.setTag('ERROR', err)
next(err)
}

parentSpan.finish()
})

const server = app.listen(process.env.PORT || 3000, () => {
console.log(`Listening on ${ server.address().address }:${ server.address().port }`)
})
})

function fakeFetch(msDelay, successRate) {
return new Promise((resolve, reject) => {
setTimeout(() => {
if (Math.random() <= successRate) {
resolve()
} else {
reject('Fake fetch failed randomly.')
}
}, msDelay)
})
}
113 changes: 113 additions & 0 deletions api-server/src/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions api-server/src/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@
"author": "Uzziah Eyee",
"license": "ISC",
"dependencies": {
"axios": "^0.18.0",
"dotenv": "^6.0.0",
"express": "^4.16.3",
"jaeger-client": "^3.14.4",
"uuid": "^3.3.2"
}
}
28 changes: 27 additions & 1 deletion docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,16 @@ services:
build: api-server
env_file: api-server/.env
environment:
- PORT=3000
#todo: change syntax to 'var: val'
PORT: 3000
JAEGER_SERVICE_NAME: api-server
# https://www.jaegertracing.io/docs/1.11/sampling/#client-sampling-configuration
JAEGER_SAMPLER_TYPE: const
JAEGER_SAMPLER_PARAM: 1
# by default, jaeger-client sends traces to an agent on localhost:6832
# override and send traces to http://tracing-backend:6832
JAEGER_AGENT_HOST: tracing-backend
JAEGER_AGENT_PORT: 6832
ports:
# so we can send test requests
- "3000:3000"
Expand All @@ -30,6 +39,7 @@ services:
- aparnet
depends_on:
- log-shipper
- tracing-backend

log-shipper:
build: log-shipper
Expand All @@ -38,5 +48,21 @@ services:
- "24224:24224/udp"
env_file: log-shipper/.env

tracing-backend:
# sadly, this image is poorly documented on dockerhub - no info on usage or available envars.
image: jaegertracing/all-in-one:1.11.0
environment:
# Storage Backend may be memory|elasticsearch|cassandra|kafka. This image only supports memory.
SPAN_STORAGE_TYPE: memory
# TODO: explicitly specify Agent port for (accepting traces 6832) (sampling configs 5778)
# receiving port of Collector over TChannel protocol
COLLECTOR_PORT: 14267
# receiving port of Query & UI
QUERY_PORT: 16686
ports:
- "16686:16686"
networks:
- aparnet

networks:
aparnet:

0 comments on commit cb6a189

Please sign in to comment.