Multi-tenant isolation in AI applications is not just about filtering database queries.
There are several boundaries to consider:
- Can one tenant retrieve another tenant's documents?
- Can one tenant consume shared cache resources unfairly?
- Can a tenant's activity evict another tenant's cached entries?
- Can application code accidentally perform operations without a tenant scope?
We recently released two RagLeap library updates addressing separate parts of this problem.
-
ragleap-optimizer 0.1.1— cache fairness and value-size protection. -
ragleap-rag 0.16.0— tenant-scoped RAG operations.
This post covers the implementation changes, verification and limitations.
1. Optimizer 0.1.1: Tenant-aware cache fairness
The original issue
The ExactCache implementation already included tenant scope in its cache key. Cached answers were therefore tenant-isolated.
However, eviction used a global LRU policy.
Consider two tenants:
-
acmehas three useful cached entries. -
globexgenerates many new entries.
Under a global LRU policy, Globex's activity could evict Acme's entries.
This is a resource-fairness problem, even though the cache keys themselves are tenant-scoped.
The fix
In 0.1.1, fair eviction is the default.
The cache also supports:
max_entries_per_scopemax_bytes_per_scopemax_bytesmax_value_bytesstats(scope)erase(scope)
The previous eviction policy remains available:
# Restore the previous LRU behaviour
cache = ExactCache(eviction="lru")
Value-size protection
The previous release accepted unbounded serialized values.
A reproduction using five approximately 40 MB values caused around 191 MB of process memory growth.
The new default is:
max_value_bytes = 1024 * 1024
Values exceeding the configured limit are refused by put(), which returns False.
Accepted values return True.
Non-serializable values raise InvalidRequestError.
To restore the previous unrestricted behaviour:
max_value_bytes = None
The important distinction: this is a serialized-value size guard, not an exact memory limit. Actual memory usage can be higher than JSON size.
Verification
- 217 tests passed on Python 3.10.12.
- 217 tests passed on Python 3.12.3 and 3.10.20.
- 119/119 mutation checks were killed on Python 3.12.3 and 3.10.20.
- Both published versions were tested against the cache behaviours that motivated the fix.
- Published wheel files were checked against the release tag.
Limitations
This implementation is single-process. It does not provide CPU or request-rate limits. Expired entries can remain counted until accessed or evicted. The size guard should not be treated as a hard process-memory boundary.
Release: https://github.com/ragleap/ragleap-core/releases/tag/ragleap-optimizer-v0.1.1
2. RAG 0.16.0: Tenant-scoped ingestion and retrieval
The second release adds an opt-in tenant-scoped view to RagLeap RAG.
Creating a tenant view
tenant_rag = rag.for_tenant("acme")
The tenant view applies the tenant boundary to the main document workflow:
- Ingestion adds the tenant ID.
- Search includes the tenant filter.
- Session IDs are tenant-prefixed.
- Listing, deleting and updating documents are tenant-restricted.
- Untagged documents are invisible through the tenant view.
This gives application code a scoped interface rather than requiring every call site to manually remember the tenant filter.
Requiring tenant scope
For applications that want to reject unscoped calls:
rag = RagLeap(
...,
require_tenant=True,
)
With this option enabled, unscoped operations raise:
TenantRequiredError
The option is disabled by default to preserve existing application behaviour.
Protecting tenant metadata
The release also fixes a metadata issue affecting chunk fields on:
- Pinecone
- Qdrant
- Milvus
- Weaviate
Caller-provided metadata can no longer overwrite the protected tenant fields in those integrations.
Milvus filter improvements
The release includes additional handling for Milvus filters:
- String values are escaped.
- Plain identifier keys are supported.
- Boolean values are represented as
trueorfalse. - Unsupported values raise
ValueError.
Tenant IDs are restricted to 1–128 characters using letters, digits, _, ., @ and -.
Important backend upgrade requirement
If tenant views are used with Redis, OpenSearch or LanceDB backends from ragleap-vectorstores, upgrade to:
ragleap-vectorstores>=0.7.0
The 0.6.x versions of those backends either ignore tenant filters or cannot filter by tenant_id.
Do not assume tenant filtering is active on those backends without the required version.
Release: https://github.com/ragleap/ragleap-core/releases/tag/ragleap-rag-v0.16.0
3. Why these are separate security boundaries
The two releases solve different problems.
| Concern | Optimizer | RAG |
|---|---|---|
| Shared cache fairness | Yes | No |
| Per-tenant cache budgets | Yes | No |
| Cache value-size guard | Yes | No |
| Tenant-scoped ingestion | No | Yes |
| Tenant-scoped retrieval | No | Yes |
| Tenant-scoped document operations | No | Yes |
A tenant-aware retrieval layer does not automatically protect shared caches.
A fair cache does not automatically protect document access.
Treat these as separate controls in your architecture and test them independently.
4. What is not guaranteed
These releases strengthen specific behaviours. They do not establish that every RagLeap component or every deployment is fully multi-tenant secure.
For example:
- The optimizer's serialized-size limit is not an exact memory cap.
- The optimizer does not enforce CPU or request-rate limits.
- The RAG tenant view depends on compatible backend implementations.
-
require_tenant=Trueis opt-in. - Applications still need to manage tenant identity and authorization correctly.
These limitations should be considered when integrating the libraries into a larger application.
5. Try the releases
pip install --upgrade ragleap-optimizer==0.1.1
pip install --upgrade ragleap-rag==0.16.0
If using the affected Redis, OpenSearch or LanceDB vector-store backends with tenant views, also upgrade:
pip install --upgrade "ragleap-vectorstores>=0.7.0"
Links
RagLeap is an open-source, self-hosted Universal AI platform built from independently usable libraries.
We welcome issue reports, technical reviews, testing and focused contributions.
If you test these releases, feedback on real application behaviour is especially valuable.
Top comments (0)