Piggy Metrics

A simple way to deal with personal finances

This is a proof-of-concept application, which demonstrates Microservice Architecture Pattern using Spring Boot, Spring Cloud and Docker. With a pretty neat user interface, by the way.

Functional services

PiggyMetrics was decomposed into three core microservices. All of them are independently deployable applications, organized around certain business capability.

Account service

Contains general user input logic and validation: incomes/expenses items, savings and account settings.

Method	Path	Description	User authenticated	Available from UI
GET	/accounts/{account}	Get specified account data
GET	/accounts/current	Get current account data	×	×
GET	/accounts/demo	Get demo account data (pre-filled incomes/expenses items, etc)		×
PUT	/accounts/current	Save current account data	×	×
POST	/accounts/	Register new account		×

Statistics service

Performs calculations on major statistics parameters and captures time series for each account. Datapoint contains values, normalized to base currency and time period. This data is used to track cash flow dynamics in account lifetime (fancy charts not yet implemented in UI).

Method	Path	Description	User authenticated	Available from UI
GET	/statistics/{account}	Get specified account statistics
GET	/statistics/current	Get current account statistics	×	×
GET	/statistics/demo	Get demo account statistics		×
PUT	/accounts/{account}	Create or update time series datapoint for specified account

Notification service

Stores users contact information and notification settings (like remind and backup frequency). Scheduled worker collects required information from other services and sends e-mail messages to subscribed customers.

Method	Path	Description	User authenticated	Available from UI
GET	/notifications/settings/current	Get current account notification settings	×	×
PUT	/notifications/settings/current	Save current account notification settings	×	×

N.B.

• Each microservice has it's own database, so there is no way to bypass API and access persistance data directly.
• In this project, I use Mongodb as a primary database for each service. It might also make sense to have a polyglot persistence architecture (сhoose the type of db that is best suited to service requirements).
• Service-to-service communication is quite simplified: microservices talking using only synchronous REST API. Common practice in a real-world systems is to use combination of interaction styles. For example, perform synchronous GET request to retrieve data and use asynchronous approach via Message broker for create/update operations in order to decouple services and buffer messages. However, this brings us in eventual consistency world.

Infrastructure services

There's a bunch of common patterns in distributed systems, which could help us to make described core services work. Spring cloud provides powerful tools that enhance Spring Boot applications behaviour to implement those patterns. I'll cover them briefly.

Config service

Spring Cloud Config is horizontally scalable centralized configuration service for distributed systems. It uses a pluggable repository layer that currently supports local storage, Git, and Subversion.

In this project, I use native profile, which simply loads config files from the local classpath. You can see shared directory in Config service resources. Those config files are shared with all applications in cluster. For example, when Statistics-service requests it's configuration, Config service will response with shared/statistics-service.yml and shared/application.yml (which is shared between all client applications).

Client side usage

Just build Spring Boot application with spring-cloud-starter-config. That's it.

You now don't need any embedded properties in your application. Just provide bootstrap.yml with application name and Config service url:

spring:
  application:
    name: notification-service
  cloud:
    config:
      uri: http://config:8888
      fail-fast: true

With Spring Cloud Config, you can change app configuration dynamically.

For example, EmailService bean was annotated with @RefreshScope. That means, you can change e-mail text and subject without rebuild and restart Notification service application.

First, change required properties in Config server. Then, perform refresh request to Notification service: curl -H "Authorization: Bearer #token#" -XPOST http://127.0.0.1:8000/notifications/refresh

Also, you could use Repository webhooks to automate this process

Notes

• There are some limitations for dynamic refresh though. @RefreshScope doesn't work with @Configuration classes and doesn't affect @Scheduled methods
• fail-fast property means that Spring Boot application will fail startup immediately, if it cannot connect to the Config Service. That's very useful when we start all applications together
• There are significant security notes below

Auth service

Authorization responsibilities are completely extracted to separate server, which grants OAuth2 tokens for the backend resource services. Auth Server is used for user authorization as well as for secure machine-to-machine communication inside a perimeter.

In this project, I use Password credentials grant type for users authorization (since it's used only by native PiggyMetrics UI) and Client Credentials grant for microservices authorization.

Spring Cloud Security provides convenient annotations to make this really easy to implement from both server and client side. You can learn more about it in documentation and check configuration details in Auth Server code.

From the client side, everything works exactly the same as with traditional session-based authorization. You can retrieve Principal object from request, check user's roles and other stuff with expression-based access control and @PreAuthorize annotation.

Each client in PiggyMetrics (account-service, statistics-service, notification-service and browser) has a scope: server for backend services, and ui - for browser. So we can also protect controllers from external access, for example:

@PreAuthorize("#oauth2.hasScope('server')")
@RequestMapping(value = "accounts/{name}", method = RequestMethod.GET)
public List<DataPoint> getStatisticsByAccountName(@PathVariable String name) {
	return statisticsService.findByAccountName(name);
}

API Gateway

As you can see, there are three core services, which expose external API to client. In a real-world systems, this number can grow very quickly as well as whole system complexity. Actualy, hundreds of services might be involved in rendering one complex webpage.

In theory, a client could make requests to each of the microservices directly. But obviously, there are challenges and limitations with this option, like necessity to know all endpoints addresses, perform http request for each peace of information separately, merge the result on a client side. Another problem is non web-friendly protocols, which might be used on the backend.

Usually a much better approach is to use API Gateway. It is a single entry point into the system, used to handle requests by routing them to the appropriate backend service or by invoking multiple backend services and aggregating the results. Also, it can be used for authentication, insights, stress and canary testing, service migration, static response handling, active traffic management.

Netflix opensourced such an edge service, and now with Spring Cloud we can enable it with one @EnableZuulProxy annotation. In this project, I use Zuul to store static content (ui application) and to route requests to appropriate microservices. Here's a simple prefix-based routing configuration for Notification service:

zuul:
  routes:
    notification-service:
        path: /notifications/**
        serviceId: notification-service
        stripPrefix: false

That means all requests starting with /notifications will be routed to Notification service. There is no hardcoded address, as you can see. Zuul uses Service discovery mechanism to locate Notification service instances and also Circuit Breaker and Load Balancer, described below.

Service discovery

Another commonly known architecture pattern is Service discovery. It allows automatic detection of network locations for service instances, which could have dynamically assigned addresses because of auto-scaling, failures and upgrades.

The key part of Service discovery is Registry. I use Netflix Eureka in this project. Eureka is a good example of the client-side discovery pattern, when client is responsible for determining locations of available service instances (using Registry server) and load balancing requests across them.

With Spring Boot, you can easily build Eureka Registry with spring-cloud-starter-eureka-server dependency, @EnableEurekaServer annotation and simple configuration properties.

Client support enabled with @EnableDiscoveryClient annotation an bootstrap.yml with application name:

spring:
  application:
    name: notification-service

Now, on application startup, it will register with Eureka Server and provide meta-data, such as host and port, health indicator URL, home page etc. Eureka receives heartbeat messages from each instance belonging to a service. If the heartbeat fails over a configurable timetable, the instance will be removed from the registry.

Also, Eureka provides simple interface, where you can track running services and number of available instances: http://localhost:8761

Http client, Load balancer and Circuit breaker

Monitor dashboard

Log analysis

Centralized logging can be very useful when attempting to identify problems in a distributed environment. Elasticsearch, Logstash and Kibana stack lets you search and analyze your logs, utilization and network activity data with ease. Ready-to-go Docker configuration described in my other project.

Security

Infrastructure automation

How to run all the things?

Feedback welcome

PiggyMetrics is open source, and would greatly appreciate your help. Feel free to contact me with any questions.