Skip to main content

NoSQL Database Performance Tuning

The MangoNoSQL time-series database uses a batch write-behind architecture to efficiently persist point values to disk. When Mango is under high load, writes are queued in memory and flushed to disk by background tasks running in parallel. Proper tuning of these settings is essential for maintaining data integrity and system performance.

If you see NoSQL Data Lost Events in your system, you should adjust the performance settings described below.

Performance Settings​

Batch Process Manager Period (ms)​

How often the system moves data from the incoming queue into the batch processing tasks for writing to the database. Lower values provide more frequent flushes but increase CPU overhead.

Batch Write Behind Spawn Threshold​

This setting determines how many point values must be waiting to be written before a new Batch Write Behind task is created. For example, a threshold of 10 creates new tasks when the queue reaches 10, 20, 30 values, and so on.

Max Batch Write Behind Tasks​

The maximum number of concurrent write tasks. Each task consumes a thread from the high-priority thread pool, so ensure your high-priority pool is large enough to accommodate both data collection and write tasks.

Batch Write Inserts Per Task (Minimum and Maximum Batch Size)​

Controls how many point values each task pulls from its per-point queue for insertion in a single iteration. This value can be adjusted to ensure faster-logging points have their values written more frequently than slower-logging points.

Minimum Time to Wait Before Flushing a Small Batch​

Allows a batch to accumulate more values before flushing, while still ensuring data is written within a reasonable time window.

Point Clean Interval​

A background task periodically removes queues for disabled data points. This setting controls how often (in milliseconds) that cleanup check runs.

Status Update Period​

The rate in milliseconds at which events are raised and metrics are sampled for the Internal Metrics page.

Stale Data Period​

If a point value queue has had no new values for this period of time, it may be discarded. Shorter periods can reduce memory usage but may degrade performance if queues are frequently destroyed and recreated.

Purge Within Shards​

Controls whether purge operations delete individual samples from within a shard file, or only delete entire shards. With this setting unchecked, purges are significantly faster because whole shard files are removed. If the purge period is shorter than a single shard (approximately 25 days), individual sample deletion occurs regardless of this setting.

Understanding the Backdate System​

The NoSQL module stores all incoming point values in a memory list. Background tasks pull values from this list and insert them into the time-series store. The system tunes throughput dynamically based on load:

  • Spawn threshold -- Determines when new write tasks are created as the list grows.
  • Max tasks -- Caps the number of parallel write threads.
  • Inserts per task -- Controls how many values each task processes per iteration.

Tasks continue running and pulling values from the list until it is empty. Multiple tasks can write data for different points simultaneously because each data point has its own time series, but a single task can only write values for one point at a time in sequence.

Tuning guidance:

  • Increasing tasks beyond your disk I/O capacity provides no benefit. If your disk can only sustain the write rate of 10 tasks, having 100 tasks will not help.
  • Having too few tasks creates a bottleneck if data arrives faster than a single task can process it.
  • Monitor the Point values to be written metric on the Internal Metrics page to see if the queue is growing over time.

Data Lost Events​

A Data Lost event is raised when a batch write fails to persist data to disk. An event is raised each time this happens, and the next successful write returns any active events to normal. Events are raised at most as frequently as the configured Status update period.

Shard Information​

NoSQL data files are stored in MA_HOME/databases/mangoTSDB/ organized by point ID. Each shard file covers approximately 24.86 days (2,147,483,648 milliseconds). Shard files are named with a numeric hash of their time range (e.g., 685.data.rev).

Purge Settings​

When configuring purge settings, keep the following in mind:

  • Set purge periods to at least 30 days to allow whole-shard deletion, which is much more efficient than per-sample deletion within a shard.
  • Default purge settings (12 months) are appropriate for most systems with fewer than several hundred thousand data points.
  • On larger production systems, consider disabling automatic NoSQL backups to save disk space and I/O, instead relying on filesystem-level backup strategies.

Max Open Files​

The db.nosql.maxOpenFiles property should not exceed the maximum open file limit of your operating system. On Linux, check with ulimit -n. The Mango systemd service file sets this to 1,048,576 by default.

The NoSQL database supports distributing data across multiple drives using symbolic links. To enable this:

  1. Create a links directory inside the NoSQL database root (MA_HOME/databases/mangoTSDB/links/).
  2. Create symbolic links within this directory pointing to folders on other drives, named by shard ID.
  3. Click Reload links in the NoSQL settings to apply changes.

New data for shards matching the link names will be written to the linked locations, distributing I/O across multiple drives.

Corruption Scan​

The NoSQL settings include a corruption scan feature with configurable thread count. Parallel corruption scans can dramatically improve performance on multi-core systems. Set the thread count based on your CPU cores and data point count.

Backup Settings​

The NoSQL module supports automatic backups to zip files, including incremental backups that capture only changes since the last backup. Configure backup settings under the Mango NoSQL Configuration section of System Settings. Key settings include:

SettingDescriptionDefault
Enable auto backupSchedule periodic backupsEnabled
Backup directoryWhere backup files are storedMA_HOME/backups
Backup frequencyHow often backups runDaily
Versions to keepNumber of backups retained before deletion9
Incremental backupsOnly backup changes since last backupDisabled

The Backup now button queues an immediate backup job.