---
name: unified-enterprise-search-rag
description: >-
  Build enterprise search and Retrieval-Augmented Generation (RAG) pipelines
  that ingest documents from many sources (Google Drive, SharePoint, Notion,
  Confluence, and more) through the single Unified.to Storage and KMS APIs. Use
  when a coding task involves pulling files, pages, or knowledge-base content
  into a vector store to ground an LLM, and keeping that index fresh in real
  time.
license: MIT
metadata:
  category: kms
  api_base: https://api.unified.to
  docs: https://docs.unified.to/kms/overview
---

# Build enterprise search with RAG using the Unified.to Storage & KMS APIs

Retrieval-Augmented Generation needs fresh, normalized content from wherever a
customer keeps it. Unified.to gives you one API across many file stores
(Storage API: Google Drive, SharePoint, Box, S3…) and knowledge bases (KMS API:
Notion, Confluence…). This skill teaches an agent how to build a RAG ingestion
pipeline against these APIs.

## When to use this skill

Use this skill when the task is to:

- Ingest a customer's documents/pages into a vector database
- Build enterprise or knowledge search over connected sources
- Ground an LLM (RAG) on real, up-to-date customer data
- Keep the index current in real time as source content changes

## Architecture

```
Source apps ──► Unified.to (Storage / KMS API) ──► your ingestion worker
                                                     │
                             chunk + embed ◄─────────┘
                                                     │
                                                     ▼
                                            vector DB (Pinecone,
                                            pgvector, Weaviate, …)
                                                     │
                    user query ──► embed ──► similarity search ──► LLM answer
```

## Prerequisites

1. A Unified.to workspace with an **API key** and **Workspace ID**.
2. Activated Storage and/or KMS integrations — start in the Sandbox.
3. A **connection ID** per connected source
   (https://docs.unified.to/concepts/embedded-components).
4. An embedding model and a vector database of your choice.

Keep the API key server-side only.

## Core request pattern

```
https://api.unified.to/storage/{connection_id}/file     # files
https://api.unified.to/kms/{connection_id}/page         # knowledge-base pages
https://api.unified.to/kms/{connection_id}/space        # spaces / collections
```

Authenticate every request:

```
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json
```

Regional base URLs: `api.unified.to` (NA), `api-eu.unified.to` (EU),
`api-au.unified.to` (AU).

## Data objects

| API     | Object    | Purpose                                        |
| ------- | --------- | ---------------------------------------------- |
| Storage | `file`    | A file/document in a connected store           |
| KMS     | `space`   | A space / collection of pages                  |
| KMS     | `page`    | A knowledge-base page (title + content)        |
| KMS     | `comment` | A comment on a page                            |

### Supported operations

| Method | Path                          | Description        |
| ------ | ----------------------------- | ------------------ |
| `GET`  | `/storage/{cid}/file`         | List files         |
| `GET`  | `/storage/{cid}/file/{id}`    | Get one file       |
| `GET`  | `/kms/{cid}/space`            | List spaces        |
| `GET`  | `/kms/{cid}/page`             | List pages         |
| `GET`  | `/kms/{cid}/page/{id}`        | Get one page       |

Both objects also support `POST / PATCH / DELETE` when you need to write back.
For RAG ingestion you mostly `GET`. Check the **Feature Support** tab in
`app.unified.to`; a `501` means unsupported.

## Ingestion: page through content

```javascript
const BASE = 'https://api.unified.to';
const headers = { Authorization: `Bearer ${process.env.UNIFIED_API_KEY}` };

async function* allPages(connectionId) {
    let offset = 0;
    const limit = 100;
    while (true) {
        const res = await fetch(
            `${BASE}/kms/${connectionId}/page?limit=${limit}&offset=${offset}`,
            { headers }
        );
        const page = await res.json();
        yield* page;
        if (page.length < limit) break;
        offset += limit;
    }
}

// Ingest each page into the vector DB
for await (const doc of allPages(connectionId)) {
    const chunks = chunk(doc.content ?? '');          // your chunker
    const vectors = await embed(chunks);              // your embedding model
    await vectorDb.upsert(
        vectors.map((v, i) => ({
            id: `${doc.id}:${i}`,
            values: v,
            metadata: { source_id: doc.id, title: doc.title, url: doc.web_url },
        }))
    );
}
```

For Storage files, list with `/storage/{cid}/file`, download the file content,
extract text, then chunk/embed/upsert the same way.

## Incremental & real-time sync

Do not re-ingest everything on a schedule. Keep the index fresh cheaply:

- **Incremental:** poll with `updated_gte=YYYY-MM-DDTHH:MM:SSZ` to fetch only
  content changed since your last run.
- **Real-time:** register **webhooks** for `file` and `page` create/update/delete
  events and upsert/delete the affected vectors on each event. See
  https://docs.unified.to/reference/webhooks and
  https://docs.unified.to/concepts/virtual_webhooks.

Always store the source `id` in vector metadata so updates and deletes map back
to the right vectors.

## Official SDKs

```bash
npm install @unified-api/typescript-sdk
```

```typescript
import { UnifiedTo } from '@unified-api/typescript-sdk';

const unified = new UnifiedTo({ security: { jwt: process.env.UNIFIED_API_KEY! } });

const { kmsPages } = await unified.kms.listKmsPages({ connectionId, limit: 100 });
```

SDKs for Python, PHP, Java, Go, C#, and Ruby: https://docs.unified.to/reference/sdks.

## Error handling

| Code | Meaning & action                                            |
| ---- | ----------------------------------------------------------- |
| 401  | Connection broken — re-authorize the end-user.              |
| 403  | Missing scopes — fix provider app + Unified.to scopes.      |
| 429  | Provider rate limit — back off; prefer webhook sync.        |
| 501  | Operation unsupported by source — check Feature Support.    |

## References

- KMS API overview & data model: https://docs.unified.to/kms/overview
- Storage API overview & data model: https://docs.unified.to/storage/overview
- Webhooks & virtual webhooks: https://docs.unified.to/reference/webhooks
- REST basics: https://docs.unified.to/reference/rest
- Source guides: https://docs.unified.to/guides/how_to_build_enterprise_search_using_rag and https://docs.unified.to/guides/retrieval_augmented_generation_rag