CDN Service

About

CDN Service is a high-performance, cloud-native content delivery solution built with Go. It provides a robust and scalable platform for managing and delivering digital assets across multiple cloud providers. The service combines modern architectural patterns with enterprise-grade features to offer:

• Multi-Cloud Storage: Seamless integration with MinIO and AWS S3, including Glacier support for cost-effective archival
• Advanced Image Processing: Real-time image resizing and optimization with support for various formats
• Enterprise Features: Circuit breaker pattern, rate limiting, and batch operations
• High Performance: Redis caching, worker pools, and optimized file handling
• Real-Time Monitoring: WebSocket-based live monitoring, Prometheus metrics, and comprehensive health checks
• Developer-Friendly: Swagger documentation, standardized API responses, and easy deployment options

Perfect for organizations needing a reliable, scalable, and feature-rich content delivery solution with support for multiple cloud providers and advanced monitoring capabilities.

Features

Storage

• Multi-cloud storage support (MinIO, AWS S3)
• Glacier archive support
• Bucket management
• Automatic file type detection
• Secure file handling

Image Processing

• Real-time image resizing
• Batch processing capabilities
• Worker pool for concurrent operations
• Support for multiple image formats
• URL-based image processing

Performance

• Redis caching layer with optimized storage
• Batch processing with configurable sizes
• Worker pool for parallel processing
• Rate limiting and request throttling
• Performance metrics and monitoring
• Dynamic ImageMagick version management
• Circuit breaker pattern for fault tolerance
  • Automatic failure detection
  • Graceful service degradation
  • Self-healing capabilities
  • Configurable thresholds and timeouts
  • Real-time state monitoring

Security

• Token-based authentication
• CORS configuration
• Rate limiting per endpoint with bypass protection
• Redis-based rate limit storage
• Request size limitations
• Trusted proxy support

Monitoring & Observability

• Prometheus metrics
• Jaeger tracing integration
• Structured logging with zerolog
• Health check endpoints
• Detailed error tracking
• Real-time system monitoring via WebSocket
• Live performance metrics
  • Active uploads count
  • Upload speed
  • Cache hit rate
  • CPU usage
  • Memory usage
  • Disk usage by mount point
  • Recent error logs

Additional Features

• Environment variable configuration
• Hot reload for configuration changes
• Swagger documentation
• Docker support
• Graceful shutdown

API Standardization

• Consistent response formats across all endpoints
• Detailed error messages and codes
• Standardized success/error patterns
• Comprehensive request validation

Quick Start

Prerequisites

• Go 1.22+
• Docker and Docker Compose
• MinIO Server (or AWS S3 access)
• Redis Server

Installation

1. Clone the repository:

git clone https://github.com/mstgnz/cdn.git
cd cdn

2. Copy the example environment file:

cp .env.example .env

3. Start the services using Docker Compose:

docker-compose up -d

Configuration

Edit the .env file with your settings:

APP_PORT=9090
APP_NAME=cdn
TOKEN=your-secure-token

# MinIO Configuration
MINIO_ENDPOINT=localhost:9000
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=minioadmin
MINIO_USE_SSL=false

# AWS Configuration (optional)
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=your-region

# Redis Configuration
REDIS_URL=redis://localhost:6379

# Feature Flags
DISABLE_DELETE=false
DISABLE_UPLOAD=false
DISABLE_GET=false

API Usage

Image Operations

1. Upload an image:

curl -X POST http://localhost:9090/upload \
  -H "Authorization: your-token" \
  -F "[email protected]" \
  -F "bucket=your-bucket" \
  -F "path=your/path"

2. Get an image with resizing:

# Original size
http://localhost:9090/your-bucket/image.jpg

# Resize with width
http://localhost:9090/your-bucket/w:300/image.jpg

# Resize with height
http://localhost:9090/your-bucket/h:200/image.jpg

# Resize with both
http://localhost:9090/your-bucket/w:300/h:200/image.jpg

3. Delete an image:

curl -X DELETE http://localhost:9090/your-bucket/image.jpg \
  -H "Authorization: your-token"

Bucket Operations

1. List buckets:

curl http://localhost:9090/minio/bucket-list \
  -H "Authorization: your-token"

2. Create bucket:

curl http://localhost:9090/minio/your-bucket/create \
  -H "Authorization: your-token"

Monitoring

1. Connect to WebSocket for real-time updates:

const ws = new WebSocket('ws://localhost:9090/ws');
ws.onmessage = (event) => {
    const stats = JSON.parse(event.data);
    console.log('System stats:', stats);
    // Example stats data:
    // {
    //   "timestamp": "2024-01-15T10:30:00Z",
    //   "active_uploads": 5,
    //   "upload_speed": 1048576, // bytes/sec
    //   "cache_hit_rate": 85.5,  // percentage
    //   "cpu_usage": 45.2,       // percentage
    //   "memory_usage": 60.8,    // percentage
    //   "disk_usage": {
    //     "/data": 75,           // percentage
    //     "/uploads": 45
    //   },
    //   "errors": [
    //     "Failed to process image: invalid format"
    //   ]
    // }
};

2. Get current monitoring stats:

curl -H "Authorization: your-token" http://localhost:9090/monitor

Kubernetes Deployment

For production deployments, we provide comprehensive Kubernetes configurations with:

• Horizontal Pod Autoscaling (3-10 pods)
• Resource quotas and limits
• Health monitoring and readiness probes
• Load balancing strategies
• Secrets management
• Persistent volume claims

For detailed instructions, see Kubernetes Deployment Guide

Documentation

For detailed information, please refer to:

• Testing Guide
• Troubleshooting Guide
• Migration Guide
• Changelog
• Kubernetes Deployment Guide
• Contributing

License

This project is licensed under the Apache License - see the LICENSE file for details.