How to build Enterprise Search using RAG

Updated:


November 19, 2025

In this article, we will be going into detail on how to build an enterprise search application as well as a Q&A bot using a RAG pipeline with Unified.to's integrations and OpenAI embedding models.

RAG or Retrieval-Augmented-Generation is an approach for indexing a large amount of information so it can be searched quickly. This design pattern is very beneficial for enterprise search, Q&A bots, or any agent. A Q&A bot can be used to answer questions across a company's knowledge base, while an enterprise search application can be used to search corporate information from any internal data source.

We will be using a few of Unified.to's integrations to access internal company knowledge in file storage (eg. Google Drive) and knowledge management systems (eg. Notion), embed their content using OpenAI's models, and then store them in a vector DB and used to answer questions.

Prerequisites

  • Node.js 19+
  • Unified account with integrations enabled for Google Drive and Notion
  • Unified.to API key and GenAI Connection ID
  • OpenAI API key (As we are using OpenAI for embedding)
  • A vector DB (eg. Pinecone, or Chroma. For the demo we are using a in memory vector store)

Step 1: Setting your project up

mkdir rag-bot
cd rag-bot
npm init -y
npm install dotenv @unified-api/typescript-sdk

Add your credentials and keys to the .env:

UNIFIED_API_KEY=your_unified_api_key
CONNECTION_GOOGLEDRIVE=your_gdrive_connection_id
CONNECTION_NOTION=your_notion_connection_id
CONNECTION_GENAI=your_genai_connection_id

Step 2: Initialize the SDK

import 'dotenv/config';
import { UnifiedTo } from '@unified-api/typescript-sdk';

const { UNIFIED_API_KEY, CONNECTION_GOOGLEDRIVE, CONNECTION_NOTION, CONNECTION_GENAI } = process.env;

const sdk = new UnifiedTo({
  security: { jwt: UNIFIED_API_KEY! },
});

Step 3: Fetch all the files from Google Drive

We will now use Unified's storage API to retrieve files from a Google Drive.

async function getAllGoogleDriveFiles(connectionId: string) {
  const files: any[] = [];
    let offset = 0;
    const limit = 100;
    let hasMore = true;
    while (hasMore) {
          const response = await sdk.storage.listStorageFiles({
            connectionId,
            offset,
            limit,
          });
            pages.push(...(response.files || []));
          hasMore = (response.files?.length || 0) === limit;
          offset += limit;
        }

      return files;
    }

    const file = await sdk.storage.getStorageFile({ connectionId, id: fileId });

    const downloadUrl = file.download_url;

    const content = await fetch(downloadUrl).then(res => res.text());

    return content;
}

Step 4: Fetch Notion Pages

Now that we have the data from Google Drive, we will use Unified's connection to Notion to fetch data from Notion as well.

async function getAllNotionPages(connectionId: string) {
  const pages: any[] = [];
    let offset = 0;
    const limit = 100;
    let hasMore = true;
    while (hasMore) {
      const response = await sdk.storage.listStorageFiles({
        connectionId,
        offset,
        limit,
      });
      files.push(...(response.files || []));
      hasMore = (response.files?.length || 0) === limit;
      offset += limit;
    }

  return pages;
}

async function getNotionPageContent(connectionId: string, pageId: string) {
  const page = await sdk.kms.getKmsPage({
    connectionId,
    id: pageId,
  });

 const content = await fetch(page.download_url).then(r => r.text());
 return content;
}

Step 5: Building the vector DB

For the purpose of the demo, we will be creating a simple in memory vector store. But for production, you would use a cloud-based vector database like Pinecone, Chroma or Weaviate.

type Document = { id: string; source: string; content: string; embedding?: number[] };
const vectorDB: Document[] = [];

Step 6: Embedding the documents

For the purpose of the example, we will be using OpenAI as the embedding provider in this step. Unified.to's Generative AI API supports embeddings on several integrations, such as OpenAI, Cohere and Google Gemini, but you can also use one provider's SDK. Please remember to chunk the content properly and not embed the entire content at once.

Chunking strategies:

Chunking large documents effectively is important to maintain context and relevance. Here are a few best practices for chunking:

  1. Semantic Chunking: Instead of splitting by fixed lengths, break the document into chunks based on natural language boundaries, like paragraphs or sections, ensuring each chunk is coherent and meaningful.
  2. Overlap Between Chunks: Include a slight overlap between adjacent chunks. This helps maintain continuity and context, especially for complex ideas that span multiple chunks.
  3. Fixed-Length Chunking: If you use fixed-length chunking, aim for a length that balances between too small (losing context) and too large (losing efficiency). Typically, a few hundred tokens per chunk works well.
  4. Using Document Structure: If your document has headings, subheadings, or other structural elements, use those to guide the chunking. This ensures that each chunk is logically complete.
  5. Contextual Embeddings: Sometimes, you can embed the entire document first and then create embeddings for chunks that are derived from the document's embedding, ensuring that each chunk stays contextually relevant.
  6. Pre-processing and Cleaning: Before chunking, clean up the text by removing unnecessary whitespace, special characters, or any irrelevant information to improve the quality of the embeddings.

By following these strategies, you'll maintain the semantic integrity of the document and ensure that the vector database can retrieve meaningful context.

There are several popular libraries that help with chunking and embedding with less custom code. A few notable ones include:

  1. LangChain: This is a popular framework that integrates various language models and vector databases. It provides built-in utilities for chunking, embedding, and storing data in vector stores.
  2. Haystack: Developed by deepset, Haystack is another framework for building search pipelines. It supports intelligent chunking, document retrieval, and integration with vector databases like FAISS, Milvus, and Pinecone.
  3. Transformers and Datasets from Hugging Face: Hugging Face offers tools that can help with both embedding and chunking. You can use their Transformers library for embedding and their Datasets library to preprocess and chunk documents.
  4. SentenceTransformers: This library is great for generating embeddings and can be combined with custom chunking logic to break documents into meaningful pieces before embedding.
  5. LlamaIndex (formerly GPT Index): This library is designed to help build large language model applications and provides utilities for chunking, indexing, and querying documents effectively.

These libraries often provide a lot of built-in functionality, making it easier to handle complex documents and ensure that your embeddings and vector storage are efficient and meaningful.

import { Configuration, OpenAIApi } from "openai";
const openai = new OpenAIApi(new Configuration({ apiKey: process.env.OPENAI_API_KEY }));

async function embedText(text: string): Promise<number[]> {
  const response = await openai.createEmbedding({
    model: "text-embedding-3-large", //You can choose to use text-embedding-3-small too
    input: text,
  });
  return response.data.data[0].embedding;
}

Step 7: Indexing Everything

Now that we have all the data collected, it is time to index all of it!

When you store vectors in a vector database like Pinecone or Weaviate, you can attach metadata to each vector. This metadata can include things like a unique ID, the original text, source information, or any other relevant attributes.

When you perform a similarity search and get the top matching vectors, the database returns the metadata along with the vectors. That way, you can easily link back to the original content.

For instance, when you index a piece of text, you'd store the vector along with metadata like a document ID, the title, or even the source URL. When you retrieve the nearest neighbors, the metadata is returned, and you can use it to identify or fetch the original content from your database or any external source.

async function indexAllContent() {
  // Google Drive
  const gdriveFiles = await getAllGoogleDriveFiles(CONNECTION_GOOGLEDRIVE!);
  for (const file of gdriveFiles) {
    const content = await getFileContent(CONNECTION_GOOGLEDRIVE!, file.id);
    const embedding = await embedText(content);
    vectorDB.push({
            id: file.id,
            source: "gdrive",
            content,
            embedding,
            filename: file.name,
            date_created: file.created_at,
            owner_id: file.user_id, 
        });
    }

  // Notion
  const notionPages = await getAllNotionPages(CONNECTION_NOTION!);
  for (const page of notionPages) {
    const content = await getNotionPageContent(CONNECTION_NOTION!, page.id);
    const embedding = await embedText(content);
    vectorDB.push({ id: page.id, source: "notion", content, embedding });
  }
}

Step 8: Finding Answers

Now that we have the data ready, it is time to implement the Bot or enterprise search that will use this data to answer questions from a user.

The basic flow would be:

  1. Embed the question asked
  2. Find similar docs in the Vector DB
  3. Send the top n amount of docs as context paired with the question to Unified's GenAI endpoint.
  4. Return the answer to the user.

For the purpose of the example, we will be sending the top 3 most similar docs.

function cosineSimilarity(a: number[], b: number[]) {
  const dot = a.reduce((sum, ai, i) => sum + ai * b[i], 0);
  const normA = Math.sqrt(a.reduce((sum, ai) => sum + ai * ai, 0));
  const normB = Math.sqrt(b.reduce((sum, bi) => sum + bi * bi, 0));
  return dot / (normA * normB);
}

async function answerQuestion(question: string) {
  const questionEmbedding = await embedText(question);

  // For the purpose of the example we will be sending the top 3 docs
  const scored = vectorDB.map(doc => ({
    ...doc,
    score: cosineSimilarity(questionEmbedding, doc.embedding!)
  }));
  const topDocs = scored.sort((a, b) => b.score - a.score).slice(0, 3);

  const context = topDocs.map(doc => doc.content).join('\n---\n');
 
  const prompt = `
You are a helpful company knowledge bot. Use the following context to answer the question.

Context:
${context}Question:
${question}Answer in a concise and accurate way.
  `;

  const result = await sdk.genai.createGenaiPrompt({
    connectionId: CONNECTION_GENAI!,
    prompt: {
      messages: [{ role: "USER", content: prompt }],
      maxTokens: 256,
      temperature: 0.2
    }
  });

  return result.choices?.[0]?.message?.content || "No answer generated.";
}

Step 9: Usage

(async () => {
  await indexAllContent();
  const answer = await answerQuestion("How do I request vacation days?");
  console.log("Bot answer:", answer);
})();

The same flow works across industries: Unified.to supports 39 categories and 2300 integrations to read end-customer data from.

Share
All guidesAccounting Account Type Hierarchy in UnifiedAdvertising Report Metrics by IntegrationAmazon S3: Required IAM PermissionsATS to Vector DB: How to Power Talent Intelligence with Real-Time DataBuilding AI applications with Unified and LangbaseConcur & Concur (Company) — Connection GuideConfigure Greenhouse Native Candidate WebhooksConfigure Salesforce Meetings and Notes for UnifiedConnect Amazon Seller Central to Unified.toConnecting BambooHR via OAuth 2Connecting Google Workspace Integrations with a Service AccountCorrect WelcomeKit Scopes for Jungle IntegrationCreating Ads using the Unified Ads APICreating Walmart Items via UnifiedEnd-Users, Integrations, and ConnectionsEnriching Unified Objects with Custom MetadataEnterprise-managed authorization (EMA) with a static keyEnterprise-managed authorization (EMA) with a workspace secretEnterprise-managed authorization (EMA) with Microsoft Entra IDExtended observability — pushing API logs to ClickHouseExtended observability — pushing API logs to Grafana / LokiExtended Observability - Pushing API Logs to your Datadog InstanceFacebook Messenger Webhook SetupFireflies Integration GuideGetting started with WorkableHandling Delegated vs. Application Scopes in Microsoft IntegrationsHiBob & Unified: connection and time off guideHow long are logs retainedHow to access employees and usersHow to add API support for the Create Activity in CrelateHow to Associate a Connection ID with Your End-UserHow to build a Candidate Assessment product withHow to build a candidate sourcing or job board app with Unified.toHow to build a Discord support bot with Unified.to and LangbaseHow to Build a Fintech Application with Unified's Payments APIHow to Build an E-Commerce Product Integration with UnifiedHow to build an invoicing system with Unified.toHow to configure Bullhorn Redirect URI for creating a connectionHow to configure webhooks in HubSpotHow to configure webhooks in Thrive LearningHow to configure webhooks in Toast for UnifiedHow to Connect Anthropic to Real-Time SaaS Data with Unified.to MCP ServerHow to Connect Cohere to Real-Time SaaS Data with Unified.to MCP ServerHow to Connect Google Gemini to Real-Time SaaS Data with Unified.to MCP ServerHow to connect HiBob (Bob) to Unified with OAuth 2.0How to Connect LLMs to Real-Time SaaS Data with Unified.to MCP ServerHow to Connect OpenAI to Real-Time SaaS Data with Unified.to MCP ServerHow to Connect Zoom and Zoom PhoneHow to create a Connection In Microsoft TeamsHow to create a token-based connection in HighlevelHow to Create a Unified.to Connection to HighLevelHow to Create a Xero Connection in Unified.toHow to create and configure webhooksHow to create Connection with HubspotHow to customize portal URLs for Stripe and GoCardlessHow to filter webhook eventsHow-to Get a Private Support Channel for Your Unified IntegrationHow to get help fromHow to get your 8x8 Connect API Key, Account ID and SMS Subaccount ID (optional): Step-by-step guideHow to get your ADP Workforce Now Oauth2 client ID, Oauth2 client secret, Oauth2 PEM certificate and Oauth2 private key: Step-by-step guideHow to get your Amazon S3 AWS Region, AWS S3 Key, AWS S3 Secret and AWS S3 Bucket(s) (optional, comma-separated): Step-by-step guideHow to get your Ashby API Key: Step-by-step guideHow to get your Brex API token: Step-by-step guideHow to get your Bullhorn OAuth2 Client ID, OAuth2 Client Secret, API Username and API Password: Step-by-step guideHow to get your Ceridian Dayforce Username, Password and Client Namespace: Step-by-step guideHow to get your Crelate API key: Step-by-step guideHow to get your Dashlane API Key: Step-by-step guideHow to get your Deel Access Token: Step-by-step guideHow to get your Dialpad API Key: Step-by-step guideHow to get your Discord OAuth 2 credentials and bot tokenHow to get your ELMO client ID and client secret: Step-by-step guideHow to get your Gainsight Client ID, Client Secret and Gainsight API Domain: Step-by-step guideHow to get your Greenhouse API Key and Job board token: Step-by-step guideHow to get your Helpscout API Key: Step-by-step guideHow to get your HiBob Service User ID and Service User Token: Step-by-step guideHow to get your HighLevel API Key and Location ID: Step-by-step guideHow to get your HubSpot developer key and OAuth 2 credentials (Legacy Apps)How to get your Humaans API Key: Step-by-step guideHow to get your iCIMS API Username, API Password and Customer ID: Step-by-step guideHow to get your JobDiva Client ID, Username/email and Password: Step-by-step guideHow to get your Jobvite API Key, API Key Secret and Email: Step-by-step guideHow to get your Lever API Key: Step-by-step guideHow to get your Loxo Agency slug, API Key and Agency ID: Step-by-step guideHow to get your Microsoft Active Directory / Entra ID Client ID, Client Secret and Tenant ID: Step-by-step guideHow to get your Microsoft Azure AD OAuth 2 credentialsHow to get your OAuth 2 credentials for GmailHow to get your OAuth 2 credentials for Microsoft Dynamics 365How to get your OAuth 2 credentials in PipeDriveHow to get your OpenAI API Key: Step-by-step guideHow to get your Paycom SID/Username and Token/Password: Step-by-step guideHow to get your PCRecruiter Username, Password, Database ID, App Key and App ID: Step-by-step guideHow to get your SAP SuccessFactors (OpenID Connect) Username, Password, Client ID, Client Secret, Dependency Name, IAS Host and API URL: Step-by-step guideHow to get your SAP SuccessFactors Username, Company ID, Password and API URL: Step-by-step guideHow to get your Shopify Admin API access token and Store ID: Step-by-step guideHow to get your Wayfair Client ID, Client Secret and Vendor Encrypted Key: Step-by-step guideHow to get your Workable API Token and Subdomain: Step-by-step guideHow to get your yotpo APP Key / Store ID and Secret Key: Step-by-step guideHow to Handle Attachments in Invoice, Bill and Credit MemoHow to Manage Google Ad Manager Orders, Line Items and Creatives with theHow to migrate or import your integrations into Unified.toHow to obtain your GitHub OAuth2 credentialsHow to Power Claude with Live Customer Data Using Unified MCPHow to receive assessment orders from an ATS with Unified.toHow to register a Google developer app and get OAuth 2 credentialsHow to Register a HiBob Developer App and Get OAUTH2 CredentialsHow to register a Salesforce developer app and get OAuth 2 credentialsHow to register a Slack developer account and get OAuth 2 credentialsHow to Register a Workday Developer App and Get OAUTH2 CredentialsHow to Register a Workday Developer App and Get OAUTH2/ SOAP CredentialsHow to Register a ZendeskSell Developer App and Get OAUTH2 CredentialsHow to Register a Zoho Developer App and Get OAUTH2 CredentialsHow to Register an ADP Developer App and Get OAUTH2 CredentialsHow to Register an Atlassian Developer App and Get OAUTH2 CredentialsHow to Register your Facebook OAuth2 ApplicationHow to Register your Google Ads OAuth2 ApplicationHow to Register your MetaAds OAuth2 applicationHow To Request & Write Raw Integration DataHow to Retrieve Microsoft Dynamics 365 Business Central CredentialsHow to set up a custom API URL with a CNAMEHow to Set Up a Google Meet Meeting Bot with UnifiedHow to set up a HiBob marketplace installHow to Set Up a Microsoft Teams Bot with UnifiedHow to Set Up a Microsoft Teams Meeting Bot with UnifiedHow to Set Up a Webex Meeting Bot with UnifiedHow to Set Up a Zoom Meeting Bot with UnifiedHow to set up and configure NotionHow to set up Google Business Profile API access and get OAuth 2 credentialsHow to Set Up LinkedIn Webhooks with UnifiedHow to set up native webhooks for QuickBooks Online with UnifiedHow to set up Slack webhooks using event subscriptionsHow to set up Telegram (bot) with Unified.toHow to set up your scopes in HubSpotHow to setup a Freshbooks developer appHow to setup a TikTok Shop applicationHow to Setup AWS Assume Role for AWS Secret ManagerHow to setup Oracle JD EdwardsHow to Setup Quickbooks DesktopHow to setup Sage 100 with Unified.toHow to test webhooks in the Synthetic Data (Sandbox) environmentHow to troubleshoot unhealthy connectionsHow to troubleshoot unhealthy webhooksHow to Use Decision Models with the Unified GenAI APIHow to use SCIM with the Unified APIHow to use the Passthrough APIHow to Use Unified.to's Generative AI API with OpenAI and ClaudeHow to use User Provisioning and Verification with UnifiedHow virtual webhook retry and recovery worksIntegration set-up guide for LeverManaging custom validation rules in SalesforceNetSuite Authentication Setup for UnifiedNotice of Deprecation of Fields and Objects (Nov 2025)Oracle HCM / Taleo OAuth Credentials GuideRetrieval-Augmented Generation (RAG)SalesForce (External Client Apps) on Multiple OrganizationsSAML Single-Sign-OnScaling MCP Tools with Anthropic's (& OpenAI's) Defer LoadingScaling MCP Tools with Anthropic's Programmatic Tool CallingSet Environments for your Unified.to WorkspaceSetting up a Slack Bot Connection with Unified.toSetting up OAuth 2 Credentials for Greenhouse's APIsTrello Connection Guide in UnifiedUnderstanding OAuth2 Authorization flowsUnderstanding Response Time Headers in the Unified APIUnified Assessment API: Embed Assessments Inside ATS PlatformsUnified's MCP ServerUnified MCP Server with LangChain and LangGraphUnlock real-time data with virtual webhooksUse Unified to sign in your users into your applicationVincere OAuth Setup Guide for UnifiedWhy you can't use Unified's OAuth2 credentials to authorize your end users on ProductionWorkable: Creating candidates and attaching them to jobsWorkday vs Workday Legacy — Why Unified.to Supports BothWorking with hierarchical tree data in Storage, Messaging, and KMS APIs
Are we missing anything? Let us know
Was this page helpful?