Database Limits

Appmixer 6.4 introduces a comprehensive database resource management system that controls storage consumption, flow resource usage, and overall system capacity. The database limits feature provides three-layer protection: per-user storage quotas, flow-level resource limits, and system-wide capacity management.

This feature is disabled by default to allow time for capacity planning and configuration before enabling in production environments.

Overview

The database limits system implements a multi-tier quota design with three main components:

  1. Storage Quotas: Per-user database storage limits with guaranteed, soft, and hard thresholds

  2. Flow Limits: Resource constraints on flows, components, and polling operations

  3. System Capacity Protection: Circuit breaker that prevents system-wide resource exhaustion

Three-Tier Storage System

Appmixer uses a three-tier storage quota system designed to support overcommitment. This allows efficient capacity utilization while protecting both users and the system.

Tier 1: Guaranteed Quota (e.g., 10MB per user)

  • Reserved capacity used for capacity planning

  • Formula: Max users = Physical capacity / Guaranteed per user

  • Example: 20GB physical ÷ 10MB guaranteed = 2,000 users safely supported

Tier 2: Soft Limit (e.g., 100MB per user)

  • Allows "burst" capacity beyond guaranteed allocation

  • Warnings sent when approaching (default 90%)

  • Supports overcommitment (10x more users possible)

  • Most users stay well below this threshold

Tier 3: Hard Limit (e.g., 500MB per user)

  • Absolute maximum, emergency brake

  • User operations blocked when exceeded

  • Very few users expected to reach this level

This design allows a 20GB system to safely support 2,000 users (guaranteed planning) while potentially handling 10-20x more users in practice due to actual average usage being well below soft limits.

Storage Quotas

Storage quotas control per-user database consumption across all collections (flows, accounts, stores, messages, etc.). When enabled, Appmixer tracks each user's storage and enforces limits to prevent resource monopolization.

Enable Storage Quotas

STORAGE_QUOTA_ENABLED

Enable or disable the storage quota system.

Default value: false (disabled)

Example:

Default Per-User Limits

STORAGE_QUOTA_DEFAULT_GUARANTEED

Guaranteed bytes reserved per user for capacity planning. Used to calculate maximum safe user count.

Default value: 10485760 (10 MB)

STORAGE_QUOTA_DEFAULT_SOFT_LIMIT

Soft limit in bytes. When exceeded, warnings are sent but operations continue.

Default value: 104857600 (100 MB)

STORAGE_QUOTA_DEFAULT_HARD_LIMIT

Hard limit in bytes. When exceeded, user operations are blocked.

Default value: 524288000 (500 MB)

STORAGE_QUOTA_DEFAULT_WARNING_THRESHOLD

Percentage of soft limit at which warnings begin.

Default value: 0.9 (90%)

Example - custom user limits:

Storage Status Levels

The system tracks four status levels based on usage:

STATUS_OK: usage < (softLimit × warningThreshold)

  • All operations allowed

  • No notifications sent

STATUS_WARNING: usage >= (softLimit × warningThreshold) && usage < softLimit

  • All operations still allowed

  • Warning notification sent (respects 24h cooldown)

  • Message: "You're using 90%+ of your storage quota"

STATUS_SOFT_EXCEEDED: usage >= softLimit && usage < hardLimit

  • All operations currently allowed

  • Notification sent: "You've exceeded your soft storage limit"

  • User should reduce storage

  • Future versions may block new flow creation

STATUS_HARD_EXCEEDED: usage >= hardLimit

  • Write operations through the system are blocked

  • Jobs are paused (polling, retries, timeouts)

  • Incoming webhooks are rejected

  • Notification sent: "Storage quota exceeded"

  • User must reduce storage to resume operations

System Capacity Configuration

STORAGE_QUOTA_SYSTEM_CAPACITY

Total physical database capacity in bytes. Used for system-level capacity management and circuit breaker thresholds.

Default value: null (unlimited, capacity protection disabled)

Example - 100GB system:

STORAGE_QUOTA_MAX_UTILIZATION

Maximum allowed system utilization percentage before circuit breaker activates and blocks all operations.

Default value: 0.95 (95%)

STORAGE_QUOTA_WARNING_UTILIZATION

System utilization percentage at which warning notifications are sent to administrators.

Default value: 0.8 (80%)

Notifications

STORAGE_QUOTA_NOTIFICATIONS_ENABLED

Enable or disable storage quota notifications (system webhooks and alerts).

Default value: true

This configuration is also available in the Appmixer Backoffice under System Configuration.

STORAGE_QUOTA_NOTIFICATION_COOLDOWN

Minimum time in milliseconds between repeat notifications for the same user and status level.

Default value: 86400000 (24 hours)

This configuration is also available in the Appmixer Backoffice under System Configuration.

Enforcement Actions

When a user exceeds their hard limit, the following operations are blocked:

STORAGE_QUOTA_BLOCK_FLOW_MODIFICATION

Block flow creation and updates when hard limit exceeded.

Default value: true

STORAGE_QUOTA_BLOCK_FLOW_TRIGGERS

Block webhook and HTTP request triggers when hard limit exceeded.

Default value: true

STORAGE_QUOTA_PAUSE_POLLING

Pause polling components when hard limit exceeded.

Default value: true

STORAGE_QUOTA_BLOCK_RETRIES

Block retry message processing when hard limit exceeded.

Default value: true

STORAGE_QUOTA_BLOCK_TIMEOUTS

Block timeout message processing when hard limit exceeded.

Default value: true

STORAGE_QUOTA_BLOCK_MESSAGE_PROCESSING

Block all message processing when hard limit exceeded.

Default value: true

Example - less restrictive enforcement:

Flow Limits

Flow limits control resource-intensive operations like the number of flows, running flows, components per flow, and active polling components. These limits prevent users from creating overly complex flows or monopolizing system resources.

Enable Flow Limits

FLOW_LIMIT_RULES_ENABLED

Enable or disable flow limit enforcement.

Default value: false (disabled)

Example:

This configuration is also available in the Appmixer Backoffice under System Configuration.

Default Limits

Regular users (scope: "user"):

  • maxFlows: -1 (unlimited)

  • maxRunningFlows: -1 (unlimited)

  • maxComponentsPerFlow: 200

  • maxPollingComponents: 500

Admin users (scope: "admin"):

  • maxFlows: -1 (unlimited)

  • maxRunningFlows: -1 (unlimited)

  • maxComponentsPerFlow: 200

  • maxPollingComponents: 1000

A value of -1 means unlimited. Flow limits use scope-based rules - if a user has multiple scopes, the rule with the highest limits applies.

Limit Types

maxFlows

  • Total number of flows (running + stopped) a user can create

  • Checked when creating new flows

  • Error: "Flow limit exceeded. You have reached the maximum of X flows."

maxRunningFlows

  • Maximum number of flows that can be running simultaneously

  • Checked when starting a flow

  • Error: "Running flow limit exceeded. You have reached the maximum of X running flows."

maxComponentsPerFlow

  • Maximum number of components allowed in a single flow

  • Checked when creating or updating flows

  • Error: "Component limit exceeded. Flow has X components but the maximum is Y."

maxPollingComponents

  • Maximum number of polling components across all running flows

  • Checked when starting a flow

  • Error: "Polling component limit exceeded. You have X polling components but the maximum is Y."

Custom Flow Limit Rules

Flow limits are configured through the FLOW_LIMIT_RULES environment variable. Rules are scope-based, allowing different limits for different user tiers.

Format: JSON array of rule objects

Example configuration:

Via environment variable:

Flow limit rules can also be configured dynamically through the Backoffice System Configuration.

System Capacity Protection

The system capacity circuit breaker monitors overall database usage and automatically blocks write operations when capacity reaches critical levels. This prevents system-wide outages due to storage exhaustion.

Enable Circuit Breaker

SYSTEM_CAPACITY_CIRCUIT_BREAKER_ENABLED

Enable or disable the system capacity circuit breaker.

Default value: false (disabled)

Example:

Circuit Breaker States

The system operates in three states based on storage utilization:

GREEN State (< 80% utilization)

  • All operations allowed

  • Normal system operation

YELLOW State (80% - 95% utilization)

  • All operations still allowed

  • Warning notification sent to administrators

  • Provides early warning for proactive action

RED State (≥ 95% utilization)

  • Write operations blocked across entire system

  • Read operations continue

  • Administrator notification sent

  • Error returned: "System at capacity (X% used)"

Monitoring Configuration

SYSTEM_CAPACITY_NOTIFICATION_COOLDOWN

Minimum time in milliseconds between capacity notifications.

Default value: 3600000 (1 hour)

This configuration is also available in the Appmixer Backoffice under System Configuration.

Controlled Subsystems

Each subsystem can be individually enabled or disabled for circuit breaker control:

SYSTEM_CAPACITY_CB_INPUT_QUEUE

Block message processing when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_DISPATCHER

Block message dispatching when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_WEBHOOKS

Block webhook endpoint execution when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_FILE_UPLOAD

Block file uploads when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_FLOW_CREATE

Block flow creation when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_FLOW_START

Block flow starting when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_USER_SIGNUP

Block new user registration when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_DATA_STORE

Block data store write operations when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_POLLING

Skip PollingController job execution when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_DELAYED_MESSAGES

Skip DelayedMessages (retry) job when circuit breaker is RED.

Default value: true

SYSTEM_CAPACITY_CB_COMPONENT_TIMEOUTS

Skip ComponentTimeouts job when circuit breaker is RED.

Default value: true

Monitoring & Reporting

Appmixer provides several APIs and tools for monitoring storage usage and managing limits.

User-Facing APIs

GET /quota/storage

Returns current user's storage usage and limits.

Response:

GET /quota/flows

Returns current user's flow limits and usage.

Response:

Admin APIs

POST /system/storage-usage-report

Generates comprehensive storage usage report for all users. Returns a ticket for polling the result.

Request:

Response:

GET /system/storage-usage-report/{ticket}

Retrieves the storage report results.

Response:

POST /system/flow-limits-violations-report

Generates flow limits violations report.

GET /system/health

Returns system health information including storage capacity stats.

Response:

System Webhooks

Storage quota events trigger system webhooks that allow you to monitor and respond to storage usage across your Appmixer installation. These webhooks are configured in the Appmixer Backoffice and can be integrated with your notification systems, monitoring tools, or custom automation workflows.

Available webhook events:

  • storage_quota_warning - User reached warning threshold (default: 90% of soft limit)

  • storage_quota_soft_exceeded - User exceeded soft limit

  • storage_quota_hard_exceeded - User exceeded hard limit (operations blocked)

  • system_capacity_warning - System reached 80% capacity (YELLOW state)

  • system_capacity_critical - System reached 95% capacity (RED state)

Complete Documentation: For detailed webhook configuration, payload schemas, notification cooldown behavior, and integration examples, see the System Webhooks - Storage Quota Notifications documentation.

Configure these webhook URLs in the Appmixer Backoffice under the following variable names:

  • WEBHOOK_STORAGE_QUOTA_WARNING

  • WEBHOOK_STORAGE_QUOTA_SOFT_EXCEEDED

  • WEBHOOK_STORAGE_QUOTA_HARD_EXCEEDED

  • WEBHOOK_SYSTEM_CAPACITY_WARNING

  • WEBHOOK_SYSTEM_CAPACITY_CRITICAL

API Access

The Database Limits system provides API endpoints for querying quota information programmatically. These APIs allow you to integrate quota monitoring into your application or build custom dashboards.

GET /quota/storage

Retrieves storage quota information for the authenticated user.

Authentication: Required (user token)

Response when storage quotas are enabled:

Response when storage quotas are disabled:

Response before first calculation:

Status values:

  • STATUS_OK - Usage below warning threshold

  • STATUS_WARNING - Usage above warning threshold

  • STATUS_SOFT_EXCEEDED - Usage exceeds soft limit

  • STATUS_HARD_EXCEEDED - Usage exceeds hard limit (operations blocked)

GET /quota/flows

Retrieves flow limit information and current usage for the authenticated user.

Authentication: Required (user token)

Response when flow limits are enabled:

Response when flow limits are disabled:

Response when no rule is defined for user's scope:

Note: A limit value of -1 means unlimited.

Capacity Planning

Proper capacity planning ensures your Appmixer installation can handle expected user growth while maintaining performance and stability.

Calculate Maximum Users

Formula:

Example:

Overcommitment Strategy

In practice, most users consume far less than their soft limit. Typical usage patterns:

  • P50 (median): ~5MB per user

  • P75: ~15MB per user

  • P90: ~35MB per user

  • P95: ~75MB per user

  • P99: ~150MB per user

Realistic capacity:

Multi-Level Protection

The three-layer system provides defense in depth:

  1. User-level (Storage Quotas): Prevents individual users from consuming excessive resources

  2. Flow-level (Flow Limits): Prevents resource-intensive flow designs

  3. System-level (Circuit Breaker): Final protection against total capacity exhaustion

Small Deployment (< 100 users, 20GB capacity):

Medium Deployment (100-1000 users, 100GB capacity):

Large Deployment (1000+ users, 500GB+ capacity):

Troubleshooting

User Storage Quota Exceeded

Symptoms:

  • User cannot create or update flows

  • Webhooks not triggering

  • Polling components paused

  • Error: "Storage quota exceeded"

Resolution:

  1. User checks current usage: GET /quota/storage

  2. User deletes unnecessary data:

    • Old flows

    • Data store entries

    • Unprocessed messages

  3. Operations automatically resume within 15 minutes (next StorageQuotaTracker run)

Admin actions:

  • Increase user's hard limit via Quota Management API

  • Assign user to higher-tier scope with larger limits

Flow Limits Exceeded

Symptoms:

  • Cannot create new flows

  • Cannot start flows

  • Error: "Flow limit exceeded" or "Polling component limit exceeded"

Resolution:

  1. User checks limits: GET /quota/flows

  2. User stops unnecessary flows

  3. User reduces components in complex flows

  4. User deletes unused flows

Admin actions:

  • Increase limits for user's scope

  • Assign user to higher-tier scope

System Capacity Exceeded

Symptoms:

  • HTTP 503 Service Unavailable errors across the system

  • Error: "System at capacity"

  • All write operations blocked

  • /system/health shows RED state

Resolution:

  1. Check capacity: GET /system/health

  2. Generate storage report: POST /system/storage-usage-report

  3. Identify high-usage users

  4. Contact high-usage users to reduce storage

  5. Consider increasing physical capacity

  6. System auto-recovers when utilization drops below 95%

Prevention:

  • Set up monitoring alerts at 80% (YELLOW state)

  • Review storage reports regularly

  • Plan capacity upgrades before reaching 70%

Quota Not Updating

Symptoms:

  • User deleted data but operations are still blocked

  • Usage numbers not changing

Resolution:

  • Wait for next StorageQuotaTracker run (every 15 minutes by default)

  • Check job logs for errors

  • Verify job is running: Check jobs logs for "StorageQuotaTracker"

  • Manually trigger recalculation if needed

Migration Guide

Database limits are disabled by default to allow proper planning before enabling in production.

Step 1: Analyze Current Usage

Before enabling quotas, understand current storage consumption:

Step 2: Plan Capacity

Based on current usage and growth projections:

  1. Calculate physical capacity needed

  2. Determine guaranteed quota per user

  3. Set soft and hard limits

  4. Plan for overcommitment ratio

  5. Define scope-based rules for different user tiers

Step 3: Configure Limits

Set environment variables based on planning:

Step 4: Set Up Monitoring

Configure system webhooks for quota events in Backoffice:

  • Storage warning events

  • Storage exceeded events

  • System capacity warnings

Set up external monitoring:

  • Alert when system reaches 70% capacity

  • Daily storage usage reports

  • Track quota violations

Step 5: Communicate to Users

Before enforcement:

  1. Announce upcoming quota limits

  2. Provide current usage information

  3. Explain how to check usage (GET /quota/storage)

  4. Document how to reduce storage if needed

  5. Set grace period before hard enforcement

Step 6: Enable Gradually

Phase 1: Enable tracking without enforcement (notifications only)

Phase 2: Enable soft enforcement (warnings + flow creation blocking)

Phase 3: Full enforcement

Step 7: Monitor and Adjust

After enabling:

  • Monitor violation rates

  • Adjust limits based on actual usage patterns

  • Review reports weekly for first month

  • Tune overcommitment strategy

  • Update limits for different user tiers as needed

Database limits provide essential resource management for multi-tenant deployments. Take time to plan and configure appropriately before enabling in production.

Last updated

Was this helpful?