Rate limiting: caching backend, per-method limits, delayed DB batching

[devin-ai-integration]

Scope

• Support an external cache (e.g., Redis) for rate limit state to reduce DB pressure
• Allow per-method (GET vs MODIFY) limits
• Consider delayed/batched DB writes when needed
• Improve error/global bucket handling, translations for error messages

Acceptance criteria

• Configurable cache provider with safe fallbacks
• Clear per-method overrides, documented
• Optional batched persistence to DB where applicable

Code references

• https://github.com/erkinalp/anticensor/blob/master/src/api/middlewares/RateLimit.ts#L26-L35
• Error translation note: https://github.com/erkinalp/anticensor/blob/master/src/api/middlewares/RateLimit.ts#L136-L142

[traycerai]

Plan

Observations

I've analyzed the current rate limiting implementation in the Spacebar Discord backend. The system currently uses an in-memory Map for caching rate limit state, with commented-out database persistence. The middleware supports per-method limits (GET vs MODIFY), has event bus integration via RabbitMQ/EventEmitter, and includes i18next translation support. The configuration system is well-structured with RateLimitOptions and RouteRateLimit classes. The main areas for improvement are external cache support (Redis), better per-method configuration, delayed DB batching, and internationalized error messages.

Approach

The implementation will enhance the existing rate limiting system by:

1. Cache Abstraction: Create a pluggable cache interface with Redis and in-memory implementations
2. Enhanced Configuration: Extend rate limit options to support per-method limits and cache settings
3. Batched Persistence: Implement delayed DB writes for blocked rate limits to reduce database pressure
4. Internationalization: Add translatable error messages for rate limit responses
5. Backward Compatibility: Ensure all changes are backward compatible with existing configurations

The approach maintains the current middleware structure while adding new capabilities through dependency injection and configuration extensions.

Reasoning

I explored the codebase systematically, starting with the main rate limiting middleware to understand the current implementation. I examined the configuration system to understand how rate limits are configured, checked the database entities for persistence schema, reviewed the translation system for internationalization support, and analyzed the event system for cross-instance communication. I also investigated the existing cache usage patterns and configuration structures to understand integration points.

Proposed File Changes

📄 src/util/cache/index.ts ^(NEW) 🔗

Create a cache abstraction layer with an interface ICacheProvider that defines methods for get, set, delete, and clear operations. All methods should return Promises to support both synchronous and asynchronous implementations. Include TTL support in the set method signature.

📄 src/util/cache/InMemoryCache.ts ^(NEW) 🔗

References

• 📄 src/util/cache/index.ts ^(NEW)

Implement an in-memory cache provider that implements ICacheProvider. Use a Map internally with optional TTL support using setTimeout for expiration. This serves as the fallback when Redis is not available. Include cleanup logic to remove expired entries periodically.

📄 src/util/cache/RedisCache.ts ^(NEW) 🔗

References

• 📄 src/util/cache/index.ts ^(NEW)

Implement a Redis cache provider that implements ICacheProvider. Use the ioredis library for Redis connectivity. Include connection error handling with graceful fallback to in-memory cache. Support key prefixing for multi-tenant scenarios and proper JSON serialization/deserialization for complex objects.

📄 src/util/config/types/subconfigurations/limits/ratelimits/RateLimitOptions.ts ^(MODIFY) 🔗

Extend the RateLimitOptions interface to include optional GET and MODIFY properties for per-method rate limits. Add cache configuration options including cacheProvider, cachePrefix, and cacheTTL. Fix the typo onyIp to onlyIp. Ensure backward compatibility by making all new properties optional.

📄 src/util/config/types/subconfigurations/limits/ratelimits/CacheConfiguration.ts ^(NEW) 🔗

References

• 📄 src/util/config/types/subconfigurations/limits/ratelimits/RateLimitOptions.ts ^(MODIFY)

Create a new configuration class for cache settings including Redis connection details (host, port, password, database), connection pool settings, key prefix for rate limits, default TTL values, and fallback behavior when cache is unavailable. Include validation for required fields when Redis is enabled.

📄 src/util/config/types/subconfigurations/limits/RateLimits.ts ^(MODIFY) 🔗

References

• 📄 src/util/config/types/subconfigurations/limits/ratelimits/CacheConfiguration.ts ^(NEW)

Add cache configuration to the RateLimits class by importing and including a cache property of type CacheConfiguration. Add batch configuration options for database writes including batchSize, batchInterval, and enableBatching properties. Maintain backward compatibility with existing configuration structure.

📄 src/util/config/types/subconfigurations/limits/ratelimits/index.ts ^(MODIFY) 🔗

References

• 📄 src/util/config/types/subconfigurations/limits/ratelimits/CacheConfiguration.ts ^(NEW)

Add export for the new CacheConfiguration class to make it available for import in other configuration files.

📄 src/api/middlewares/RateLimit.ts ^(MODIFY) 🔗

References

• 📄 src/util/cache/index.ts ^(NEW)
• 📄 src/util/cache/InMemoryCache.ts ^(NEW)
• 📄 src/util/cache/RedisCache.ts ^(NEW)
• 📄 src/util/entities/RateLimit.ts
• 📄 src/api/middlewares/Translation.ts

Replace the global Cache Map with an injected cache provider from the new cache abstraction layer. Update initRateLimits function to initialize the appropriate cache provider based on configuration (Redis if available, otherwise in-memory). Convert cache operations to async/await pattern. Implement batched database writes for blocked rate limits using a queue that flushes periodically. Add internationalization support for error messages using the translation middleware. Remove TODO comments that are now addressed. Update the rateLimit function to accept enhanced configuration options including per-method limits. Ensure the event bus integration works with the new cache system.

📄 assets/locales/en/common.json ^(MODIFY) 🔗

Add rate limiting error messages to the English locale file. Include keys for RATE_LIMITED with a user-friendly message, RATE_LIMITED_GLOBAL for global rate limits, and RATE_LIMITED_RETRY_AFTER for retry timing information. Follow the existing JSON structure and naming conventions used in the file.

📄 package.json ^(MODIFY) 🔗

Add ioredis as a dependency for Redis support. Include it in the main dependencies section with an appropriate version (e.g., ^5.3.0). Also add @types/ioredis to devDependencies for TypeScript support.

📄 src/util/cache/CacheManager.ts ^(NEW) 🔗

References

• 📄 src/util/cache/index.ts ^(NEW)
• 📄 src/util/cache/InMemoryCache.ts ^(NEW)
• 📄 src/util/cache/RedisCache.ts ^(NEW)

Create a cache manager singleton that handles cache provider initialization based on configuration. Include logic to detect Redis availability, handle connection failures gracefully, and provide fallback to in-memory cache. Implement health checking for Redis connections and automatic reconnection logic. Export factory functions for creating cache instances.

📄 src/util/ratelimit/BatchProcessor.ts ^(NEW) 🔗

References

• 📄 src/util/entities/RateLimit.ts
• 📄 src/util/util/Event.ts

Create a batch processor for handling delayed database writes of rate limit violations. Implement a queue system that collects blocked rate limit events and flushes them to the database in batches. Include configurable batch size and interval settings. Handle database errors gracefully and provide retry logic. Use the existing RateLimit entity for persistence and integrate with the event system for cross-instance synchronization.

Diagram

sequenceDiagram
    participant Client
    participant RateLimit as RateLimit Middleware
    participant Cache as Cache Provider
    participant DB as Database
    participant EventBus as Event Bus
    participant BatchProcessor as Batch Processor

    Client->>RateLimit: HTTP Request
    RateLimit->>Cache: Get rate limit state
    Cache-->>RateLimit: Current state or null
    
    alt Rate limit not exceeded
        RateLimit->>Cache: Update hit count
        RateLimit->>Client: Allow request (200)
    else Rate limit exceeded
        RateLimit->>Cache: Mark as blocked
        RateLimit->>BatchProcessor: Queue blocked event
        RateLimit->>EventBus: Emit rate limit event
        RateLimit->>Client: Rate limited (429)
    end
    
    BatchProcessor->>DB: Batch write blocked events
    EventBus->>Cache: Sync state across instances

Loading

Import In IDE

🤖 Prompt for AI Agents

I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

I've analyzed the current rate limiting implementation in the Spacebar Discord backend. The system currently uses an in-memory Map for caching rate limit state, with commented-out database persistence. The middleware supports per-method limits (GET vs MODIFY), has event bus integration via RabbitMQ/EventEmitter, and includes i18next translation support. The configuration system is well-structured with `RateLimitOptions` and `RouteRateLimit` classes. The main areas for improvement are external cache support (Redis), better per-method configuration, delayed DB batching, and internationalized error messages.

### Approach

The implementation will enhance the existing rate limiting system by:

1. **Cache Abstraction**: Create a pluggable cache interface with Redis and in-memory implementations
2. **Enhanced Configuration**: Extend rate limit options to support per-method limits and cache settings
3. **Batched Persistence**: Implement delayed DB writes for blocked rate limits to reduce database pressure
4. **Internationalization**: Add translatable error messages for rate limit responses
5. **Backward Compatibility**: Ensure all changes are backward compatible with existing configurations

The approach maintains the current middleware structure while adding new capabilities through dependency injection and configuration extensions.

### Reasoning

I explored the codebase systematically, starting with the main rate limiting middleware to understand the current implementation. I examined the configuration system to understand how rate limits are configured, checked the database entities for persistence schema, reviewed the translation system for internationalization support, and analyzed the event system for cross-instance communication. I also investigated the existing cache usage patterns and configuration structures to understand integration points.

## Mermaid Diagram

sequenceDiagram
    participant Client
    participant RateLimit as RateLimit Middleware
    participant Cache as Cache Provider
    participant DB as Database
    participant EventBus as Event Bus
    participant BatchProcessor as Batch Processor

    Client->>RateLimit: HTTP Request
    RateLimit->>Cache: Get rate limit state
    Cache-->>RateLimit: Current state or null
    
    alt Rate limit not exceeded
        RateLimit->>Cache: Update hit count
        RateLimit->>Client: Allow request (200)
    else Rate limit exceeded
        RateLimit->>Cache: Mark as blocked
        RateLimit->>BatchProcessor: Queue blocked event
        RateLimit->>EventBus: Emit rate limit event
        RateLimit->>Client: Rate limited (429)
    end
    
    BatchProcessor->>DB: Batch write blocked events
    EventBus->>Cache: Sync state across instances

## Proposed File Changes

### src/util/cache/index.ts(NEW)

Create a cache abstraction layer with an interface `ICacheProvider` that defines methods for `get`, `set`, `delete`, and `clear` operations. All methods should return Promises to support both synchronous and asynchronous implementations. Include TTL support in the `set` method signature.

### src/util/cache/InMemoryCache.ts(NEW)

References: 

- src/util/cache/index.ts(NEW)

Implement an in-memory cache provider that implements `ICacheProvider`. Use a Map internally with optional TTL support using setTimeout for expiration. This serves as the fallback when Redis is not available. Include cleanup logic to remove expired entries periodically.

### src/util/cache/RedisCache.ts(NEW)

References: 

- src/util/cache/index.ts(NEW)

Implement a Redis cache provider that implements `ICacheProvider`. Use the `ioredis` library for Redis connectivity. Include connection error handling with graceful fallback to in-memory cache. Support key prefixing for multi-tenant scenarios and proper JSON serialization/deserialization for complex objects.

### src/util/config/types/subconfigurations/limits/ratelimits/RateLimitOptions.ts(MODIFY)

Extend the `RateLimitOptions` interface to include optional `GET` and `MODIFY` properties for per-method rate limits. Add cache configuration options including `cacheProvider`, `cachePrefix`, and `cacheTTL`. Fix the typo `onyIp` to `onlyIp`. Ensure backward compatibility by making all new properties optional.

### src/util/config/types/subconfigurations/limits/ratelimits/CacheConfiguration.ts(NEW)

References: 

- src/util/config/types/subconfigurations/limits/ratelimits/RateLimitOptions.ts(MODIFY)

Create a new configuration class for cache settings including Redis connection details (host, port, password, database), connection pool settings, key prefix for rate limits, default TTL values, and fallback behavior when cache is unavailable. Include validation for required fields when Redis is enabled.

### src/util/config/types/subconfigurations/limits/RateLimits.ts(MODIFY)

References: 

- src/util/config/types/subconfigurations/limits/ratelimits/CacheConfiguration.ts(NEW)

Add cache configuration to the `RateLimits` class by importing and including a `cache` property of type `CacheConfiguration`. Add batch configuration options for database writes including `batchSize`, `batchInterval`, and `enableBatching` properties. Maintain backward compatibility with existing configuration structure.

### src/util/config/types/subconfigurations/limits/ratelimits/index.ts(MODIFY)

References: 

- src/util/config/types/subconfigurations/limits/ratelimits/CacheConfiguration.ts(NEW)

Add export for the new `CacheConfiguration` class to make it available for import in other configuration files.

### src/api/middlewares/RateLimit.ts(MODIFY)

References: 

- src/util/cache/index.ts(NEW)
- src/util/cache/InMemoryCache.ts(NEW)
- src/util/cache/RedisCache.ts(NEW)
- src/util/entities/RateLimit.ts
- src/api/middlewares/Translation.ts

Replace the global `Cache` Map with an injected cache provider from the new cache abstraction layer. Update `initRateLimits` function to initialize the appropriate cache provider based on configuration (Redis if available, otherwise in-memory). Convert cache operations to async/await pattern. Implement batched database writes for blocked rate limits using a queue that flushes periodically. Add internationalization support for error messages using the translation middleware. Remove TODO comments that are now addressed. Update the `rateLimit` function to accept enhanced configuration options including per-method limits. Ensure the event bus integration works with the new cache system.

### assets/locales/en/common.json(MODIFY)

Add rate limiting error messages to the English locale file. Include keys for `RATE_LIMITED` with a user-friendly message, `RATE_LIMITED_GLOBAL` for global rate limits, and `RATE_LIMITED_RETRY_AFTER` for retry timing information. Follow the existing JSON structure and naming conventions used in the file.

### package.json(MODIFY)

Add `ioredis` as a dependency for Redis support. Include it in the main dependencies section with an appropriate version (e.g., `^5.3.0`). Also add `@types/ioredis` to devDependencies for TypeScript support.

### src/util/cache/CacheManager.ts(NEW)

References: 

- src/util/cache/index.ts(NEW)
- src/util/cache/InMemoryCache.ts(NEW)
- src/util/cache/RedisCache.ts(NEW)

Create a cache manager singleton that handles cache provider initialization based on configuration. Include logic to detect Redis availability, handle connection failures gracefully, and provide fallback to in-memory cache. Implement health checking for Redis connections and automatic reconnection logic. Export factory functions for creating cache instances.

### src/util/ratelimit/BatchProcessor.ts(NEW)

References: 

- src/util/entities/RateLimit.ts
- src/util/util/Event.ts

Create a batch processor for handling delayed database writes of rate limit violations. Implement a queue system that collects blocked rate limit events and flushes them to the database in batches. Include configurable batch size and interval settings. Handle database errors gracefully and provide retry logic. Use the existing `RateLimit` entity for persistence and integrate with the event system for cross-instance synchronization.

---

Developer Humor

Why did the rate limiter go to therapy?
Because it had too many issues with boundaries! 🚦

It kept saying You shall not pass!
But learned to say Please try again later instead.
Now it's much more Redis-onable! 📦✨

---

Execution Information

Branch: master
Commit: 0442d70

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[erkinalp]

Also needs to deal with certain endpoints that perform atomic swaps in transactional blocks