Trong comprehensive guide này, bạn sẽ build một distributed rate limiter dùng Redis và Lua scripting để control user requests trong high-traffic environment.
Rate limiting crucial trong bất kỳ system nào để prevent abuse, manage traffic, và protect resources. Bằng cách leverage Redis và Lua, bạn sẽ build efficient, scalable rate limiting system handle large number of requests đồng thời giữ backend services safe.
Chúng ta cũng include interactive demo nơi users simulate traffic, observe rate limits enforced, và view logs blocked requests.
What You Will Learn
Cách build rate limiting system dùng Redis.
Cách dùng Lua scripts với Redis achieve atomic operations.
Hiểu Redis data structures cho efficient request tracking.
Techniques handle high traffic trong distributed system.
Dùng Docker simulate và scale distributed rate limiter.
Prerequisites
Trước khi bắt đầu, ensure bạn có installed:
Node.js (v14 hoặc cao hơn)
Redis
Docker (cho simulate distributed environment)
Basic understanding Node.js, Redis, và Lua scripting.
Mục lục
What You Will Learn
Prerequisites
Project Overview
Step 1: How to Set Up the Project
Step 2: How to Set Up Redis
Step 3: How to Implement the Rate Limiter with Redis and Lua
Step 4: How to Create the Node.js API Server
Step 5: How to Test the Rate Limiter
Step 6: How to Visualize Rate Limiting Metrics
Step 7: How to Deploy with Docker
Conclusion: What You've Learned
Project Overview
Trong tutorial này, bạn sẽ:
Build rate limiter dùng Redis và Lua enforce request quotas.
Dùng Lua scripts ensure atomic operations, avoid race conditions.
Implement token bucket algorithm cho rate limiting.
Tạo interactive demo simulate high traffic và visualize rate limiting in action.
System Architecture
Build system với components sau:
API Server: Handle incoming user requests.
Redis: Store request data và enforce rate limits.
Lua Scripts: Ensure atomic updates Redis cho rate limiting.
Docker: Simulate distributed environment với multiple instances.
Step 1: How to Set Up the Project
Setup Node.js project:
mkdir distributed-rate-limiter
cd distributed-rate-limiter
npm init -y
Cài required dependencies:
npm install express redis dotenv
express: Lightweight web server framework.
redis: Interact với Redis.
dotenv: Manage environment variables.
Tạo .env file với content:
REDIS_HOST=localhost
REDIS_PORT=6379
PORT=3000
RATE_LIMIT=5
TIME_WINDOW=60
Variables define Redis host, port, rate limit (số requests allowed), và time window (giây).
Step 2: How to Set Up Redis
Trước khi dive vào code, ensure Redis installed và running. Nếu chưa có Redis, dùng Docker quick setup:
docker run -p 6379:6379 --name redis-rate-limiter -d redis
Step 3: How to Implement the Rate Limiter with Redis and Lua
Để efficiently handle rate limiting, dùng token bucket algorithm. Trong algorithm này:
- Mỗi user có "bucket" tokens.
- Mỗi request consume một token.
- Tokens refill periodically at set rate.
Đảm bảo atomicity và avoid race conditions, dùng Lua scripting với Redis. Lua scripts trong Redis execute atomically, nghĩa là không bị interrupt bởi operations khác while running.
How to Create a Lua Script for Rate Limiting
Tạo file rate_limiter.lua:
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local current = redis.call("get", key)
if current and tonumber(current) >= limit then
return 0
else
if current then
redis.call("incr", key)
else
redis.call("set", key, 1, "EX", window)
end
return 1
end
Inputs:
KEYS[1]: Redis key represent user's request count.
ARGV[1]: Rate limit (max allowed requests).
ARGV[2]: Time window (giây) cho rate limit.
Logic:
- Nếu user reached rate limit, return
0(request blocked). - Nếu user within limit, increment request count hoặc set new count với expiration nếu first request.
- Return 1 (request allowed).
- Nếu user reached rate limit, return
Step 4: How to Create the Node.js API Server
Tạo file server.js:
require('dotenv').config();
const express = require('express');
const redis = require('redis');
const fs = require('fs');
const path = require('path');
const app = express();
const client = redis.createClient({ host: process.env.REDIS_HOST, port: process.env.REDIS_PORT });
const rateLimitScript = fs.readFileSync(path.join(__dirname, 'rate_limiter.lua'), 'utf8');
const RATE_LIMIT = parseInt(process.env.RATE_LIMIT);
const TIME_WINDOW = parseInt(process.env.TIME_WINDOW);
// Middleware for rate limiting
async function rateLimiter(req, res, next) {
const ip = req.ip;
try {
const allowed = await client.eval(rateLimitScript, 1, ip, RATE_LIMIT, TIME_WINDOW);
if (allowed === 1) {
next();
} else {
res.status(429).json({ message: 'Too many requests. Please try again later.' });
}
} catch (err) {
console.error('Error in rate limiter:', err);
res.status(500).json({ message: 'Internal server error' });
}
}
app.use(rateLimiter);
app.get('/', (req, res) => {
res.send('Welcome to the Rate Limited API!');
});
const PORT = process.env.PORT;
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});
Rate Limiter Middleware:
- Retrieve client's IP address và check within rate limit dùng Lua script.
- Nếu user exceeds limit, gửi
429response.
API Endpoint:
- Root endpoint rate-limited, user chỉ access limited times trong specified window.
Step 5: How to Test the Rate Limiter
Start Redis:
docker start redis-rate-limiterRun Node.js Server:
node server.jsSimulate Requests:
Dùng
curlhoặc Postman test rate limiter:curl https://localhost:3000Gửi multiple requests rapidly xem rate limiting in action.
Step 6: How to Visualize Rate Limiting Metrics
Monitor rate limiting metrics như cache hits và blocked requests, thêm logging vào middleware trong server.js:
async function rateLimiter(req, res, next) {
const ip = req.ip;
try {
const allowed = await client.eval(rateLimitScript, 1, ip, RATE_LIMIT, TIME_WINDOW);
if (allowed === 1) {
console.log(`Allowed request from ${ip}`);
next();
} else {
console.log(`Blocked request from ${ip}`);
res.status(429).json({ message: 'Too many requests. Please try again later.' });
}
} catch (err) {
console.error('Error in rate limiter:', err);
res.status(500).json({ message: 'Internal server error' });
}
}
Step 7: How to Deploy with Docker
Containerize application chạy trong distributed environment.
Tạo Dockerfile:
FROM node:14
WORKDIR /app
COPY . .
RUN npm install
EXPOSE 3000
CMD ["node", "server.js"]
Build và Run Docker Container:
docker build -t rate-limiter .
docker run -p 3000:3000 rate-limiter
Bây giờ bạn scale rate limiter bằng cách chạy multiple instances.
Conclusion: What You've Learned
Chúc mừng! Bạn successfully built distributed rate limiter dùng Redis và Lua scripts. Throughout tutorial, bạn học cách:
Implement rate limiting control user requests trong distributed system.
Dùng Lua scripts trong Redis perform atomic operations.
Apply token bucket algorithm manage request quotas.
Monitor rate limiting metrics optimize performance.
Dùng Docker simulate scalable distributed environment.
Next Steps:
Add Rate Limiting by User ID: Extend system support rate limits per user.
Integrate with Nginx: Dùng Nginx as reverse proxy với Redis-backed rate limiting.
Deploy with Kubernetes: Scale rate limiter dùng Kubernetes cho high availability.
Happy coding!
Học code miễn phí. Chương trình open source freeCodeCamp đã giúp hơn 40.000 người có việc developer. Bắt đầu ngay