Engineering7 min

Multi-tenancy isn't optional once your agent platform has more than one customer

UnderOcean Team

Multi-tenancy means every query, every stored secret, and every permission check is scoped to an organization boundary by construction — not by convention, and not by a developer remembering to add a WHERE org_id = ... clause every time. An internal prototype can get away with a single shared database and a global API key sitting in an environment variable. A platform serving more than one customer cannot: one misconfigured query, one forgotten filter, and you've leaked one organization's data — or worse, their LLM provider API keys — to another. Here's the checklist we built UnderOcean's tenant isolation around.

The tenant hierarchy

UnderOcean's tenancy model has one primary boundary and one identity model layered on top of it:

Organization                       ← primary tenant boundary; org_id filters everything
  ├── OrganizationMember (User + role)
  ├── CustomRole                    ← org-scoped named permission set
  └── Project
        ├── Agent, Flow, KnowledgeBase, MCPServer, Gate, ApiKey
        └── ProviderConfig (project-scoped, or inherited from the org)

Notably, User is the one model in the system without org_id scoping — a person's identity is global, but their membership, role, and therefore permissions are scoped per organization through OrganizationMember. The same human can be an admin in one organization and have no access at all to another, which is exactly the property a multi-tenant platform needs.

Isolation at the database level, not just the application level

The checklist starts with the assumption that application-layer filtering will eventually have a bug — someone will forget a filter, or a new endpoint will be added without one. Row-level security policies scoped to org_id enforce isolation at the database level itself, so a missing application-layer filter fails closed rather than silently leaking a cross-tenant row. This is a deliberate belt-and-suspenders design: the application layer should always filter correctly, and the database layer guarantees it even on the day it doesn't.

Encrypted credentials, never returned

Every organization connects its own LLM, embedding, and other provider credentials — and those credentials are exactly the kind of thing that must never leak between tenants, or even back to the tenant's own frontend in plaintext. ProviderConfig.api_key_enc is envelope-encrypted at rest (AES-256-GCM) and deliberately excluded from every ProviderOut API response schema — an org's own admins see a masked placeholder, never the real key, the same as anyone outside the organization. The same encryption path covers other secrets that flow through the platform, including MCP server credentials and any secret-typed field a flow node declares.

RBAC: fixed roles, custom roles, and project-scoped grants

Access control layers on top of tenant isolation, since "which org can see this row" and "which member of that org is allowed to do what" are different questions:

  • Fixed org-level roles — owner, admin, member, viewer — each mapped to a frozen set of granular permissions (60+ of them, spanning organization, member, project, agent, flow, knowledge, MCP, gate, execution, provider, and API-key operations).
  • Custom roles — an org-scoped, admin-defined permission set that can be assigned instead of a fixed role, for teams whose access needs don't map cleanly onto owner/admin/member/viewer.
  • Project-scoped access grants — narrower permissions granted to a specific member or role within one project only, optionally restricted further to specific records (a named agent, a named knowledge base) rather than every resource of that type in the project.

Every route enforces this through a single dependency factory, never an inline role check:

@router.delete("/{agent_id}")
async def delete_agent(
    agent_id: UUID,
    user: User = Depends(require_permission(Permission.AGENT_DELETE)),
    db: AsyncSession = Depends(get_db),
):
    ...

Centralizing permission checks in one factory means there's exactly one place authorization logic can be wrong, audited, or updated — not sixty places, one per route, each a chance for a subtly different (and subtly wrong) inline check.

Milvus collections, named to prevent cross-tenant access at the storage level

Tenant isolation has to extend past the relational database into every other store a knowledge base touches. Each knowledge base's vector chunks live in their own Milvus collection, named deterministically from the org, project, and knowledge base IDs (kb_{org_id}_{project_id}_{kb_id}, hyphens stripped so the name is a valid Milvus identifier) — so isolation between tenants' vector data is enforced by the storage layout itself, not only by which collection an application query happens to ask for. Graph-based knowledge bases get an additional entity collection following the same naming discipline, and raw uploaded documents in MinIO are similarly namespaced by {org_id}/{project_id}/{kb_id}/{filename}.

None of this is visible when it's working

The entire point of getting multi-tenancy right is that end users never notice it. An org admin configuring a provider, a member running an agent, a viewer reading an execution log — none of them should ever be aware that isolation machinery exists, because it should be invisible infrastructure, not a feature they interact with. Multi-tenancy done right is invisible right up until the one day it isn't there, and that's the day it becomes the only thing that matters.

FAQ

Is application-layer filtering by org_id not enough on its own? It's necessary but not sufficient — row-level security at the database layer exists specifically to catch the case where an application-layer filter is missing or wrong, rather than depending on every query, in every endpoint, forever, being written correctly.

Can an organization's admin ever see another organization's data? No — organization membership and role are scoped per-org via OrganizationMember; a global is_platform_admin flag exists separately for cross-organization platform administration, and it's a distinct, explicitly-flagged capability, not an emergent side effect of being an admin somewhere.

How granular can project-level permissions get? Down to individual records for the five resource types that carry them (agents, flows, knowledge bases, MCP servers, gates) — a grant can be scoped to specific named resources rather than every resource of that type in a project.

Are provider API keys ever visible to anyone after they're saved? No — not to other tenants, and not even to the owning organization's own admins through the API; only a masked placeholder is ever returned once a key is stored.