RAG Knowledge Base

Overview

The RAG (Retrieval-Augmented Generation) API allows you to upload documents and create knowledge bases that your AI agents can search. Documents are uploaded to Google Cloud Storage and indexed in Vertex AI Search for semantic retrieval.

Workflow: First create a DataStore using POST /rag/datastores, then upload files to that DataStore using POST /rag/upload. Files are indexed asynchronously via background tasks.

Authentication

All RAG endpoints require API key authentication with specific scopes:

Endpoint	Required Scope
`POST /rag/datastores`	`rag:upload`
`GET /rag/datastores`	`rag:read`
`GET /rag/datastores/{id}/files`	`rag:read`
`DELETE /rag/datastores/{id}`	`rag:delete`
`DELETE /rag/datastores/{id}/files/{filename}`	`rag:delete`
`POST /rag/upload`	`rag:upload`
`GET /rag/uploads`	`rag:read`
`GET /rag/uploads/{upload_id}`	`rag:read`

Scopes Required: The rag:upload, rag:read, and rag:delete scopes must be explicitly enabled when creating API keys. They are not enabled by default.

Include your API key in the X-API-Key header:

X-API-Key: jns_live_YOUR_API_KEY_HERE

See Authentication for details.

Supported File Types

The following file types can be uploaded to RAG Datastores:

MIME Type	Extension	Description
`application/pdf`	`.pdf`	PDF documents
`application/vnd.openxmlformats-officedocument.wordprocessingml.document`	`.docx`	Microsoft Word documents
`application/vnd.openxmlformats-officedocument.presentationml.presentation`	`.pptx`	Microsoft PowerPoint presentations
`text/plain`	`.txt`	Plain text files
`text/html`	`.html`	HTML documents
`application/json`	`.json`	JSON files
`application/x-ndjson`	`.jsonl`	Newline-delimited JSON
`text/markdown`	`.md`, `.markdown`	Markdown documents

File Limits: Maximum 10MB per file, maximum 10 files per upload request.

Create DataStore

Create a new DataStore to organize uploaded documents.

Endpoint

POST https://api.junis.ai/api/external/rag/datastores

Request Body

Field	Type	Required	Description
`display_name`	string	Yes	Human-readable name for the DataStore (e.g., “Company Policies”)
`description`	string	No	Optional description of the DataStore’s purpose

Request Example

cURL

curl -X POST "https://api.junis.ai/api/external/rag/datastores" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "Company Policies",
    "description": "HR policies and employee handbook"
  }'

Python

import requests

response = requests.post(
    "https://api.junis.ai/api/external/rag/datastores",
    headers={
        "X-API-Key": "jns_live_YOUR_API_KEY_HERE",
        "Content-Type": "application/json"
    },
    json={
        "display_name": "Company Policies",
        "description": "HR policies and employee handbook"
    }
)
print(response.json())

JavaScript

const response = await fetch('https://api.junis.ai/api/external/rag/datastores', {
  method: 'POST',
  headers: {
    'X-API-Key': 'jns_live_YOUR_API_KEY_HERE',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    display_name: 'Company Policies',
    description: 'HR policies and employee handbook'
  })
});
const data = await response.json();
console.log(data);

Response

Status Code: 200 OK

{
  "success": true,
  "datastore_id": "company-policies-a1b2c3d4",
  "display_name": "Company Policies",
  "description": "HR policies and employee handbook",
  "message": "DataStore 'Company Policies' created successfully"
}

Response Fields

Field	Type	Description
`success`	boolean	Whether creation was successful
`datastore_id`	string	Unique ID for the DataStore (use this for uploads)
`display_name`	string	Human-readable name
`description`	string	DataStore description (null if not provided)
`message`	string	Success message

Error Responses

409 Conflict - DataStore Already Exists

{
  "detail": {
    "error": {
      "message": "DataStore with name 'Company Policies' already exists",
      "type": "conflict",
      "code": "datastore_exists",
      "existing_datastore_id": "company-policies-a1b2c3d4"
    }
  }
}

Upload Files

Upload files to an existing DataStore. Files are encoded as Base64 and indexed asynchronously.

Endpoint

POST https://api.junis.ai/api/external/rag/upload

DataStore Required: You must create a DataStore first using POST /rag/datastores. File uploads to non-existent DataStores will fail with a 404 error.

Request Body

Field	Type	Required	Description
`datastore_id`	string	Yes	ID of existing DataStore (from create response)
`files`	array	Yes	Array of files to upload (min: 1, max: 10)
`files[].filename`	string	Yes	Filename with extension (e.g., “policy.pdf”)
`files[].content`	string	Yes	Base64-encoded file content
`files[].content_type`	string	No	MIME type (auto-detected from extension if not provided)

Request Example

cURL

# First, encode your file as Base64
BASE64_CONTENT=$(base64 -i policy.pdf)

curl -X POST "https://api.junis.ai/api/external/rag/upload" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "datastore_id": "company-policies-a1b2c3d4",
    "files": [
      {
        "filename": "policy.pdf",
        "content": "'"${BASE64_CONTENT}"'"
      }
    ]
  }'

Python

import base64
import requests

# Read and encode file
with open("policy.pdf", "rb") as f:
    content = base64.b64encode(f.read()).decode()

response = requests.post(
    "https://api.junis.ai/api/external/rag/upload",
    headers={
        "X-API-Key": "jns_live_YOUR_API_KEY_HERE",
        "Content-Type": "application/json"
    },
    json={
        "datastore_id": "company-policies-a1b2c3d4",
        "files": [
            {
                "filename": "policy.pdf",
                "content": content
            }
        ]
    }
)
print(response.json())

Response

Status Code: 200 OK

{
  "success": true,
  "datastore_id": "company-policies-a1b2c3d4",
  "datastore_name": "Company Policies",
  "uploads": [
    {
      "upload_id": "550e8400-e29b-41d4-a716-446655440000",
      "filename": "policy.pdf",
      "status": "processing"
    }
  ],
  "message": "1 file(s) upload started"
}

Response Fields

Field	Type	Description
`success`	boolean	Whether upload was initiated
`datastore_id`	string	Target DataStore ID
`datastore_name`	string	Target DataStore display name
`uploads`	array	Status of each uploaded file
`uploads[].upload_id`	string	Unique upload ID (UUID)
`uploads[].filename`	string	Original filename
`uploads[].status`	string	Upload status (`processing`)
`message`	string	Summary message

Error Responses

404 Not Found - DataStore Not Found

{
  "detail": {
    "error": {
      "message": "DataStore not found: invalid-datastore-id. Create it first using POST /rag/datastores",
      "type": "not_found",
      "code": "datastore_not_found"
    }
  }
}

400 Bad Request - Unsupported File Type

{
  "detail": {
    "error": {
      "message": "Unsupported file extension: .exe",
      "type": "validation_error",
      "code": "unsupported_file_type"
    }
  }
}

400 Bad Request - File Too Large

{
  "detail": {
    "error": {
      "message": "File size exceeds 10MB limit",
      "type": "validation_error",
      "code": "file_too_large"
    }
  }
}

400 Bad Request - Invalid Base64

{
  "detail": {
    "error": {
      "message": "Base64 decoding failed: Invalid padding",
      "type": "validation_error",
      "code": "invalid_base64"
    }
  }
}

List DataStores

Retrieve all DataStores in your organization.

Endpoint

GET https://api.junis.ai/api/external/rag/datastores

Request Example

cURL

curl -X GET "https://api.junis.ai/api/external/rag/datastores" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE"

Python

import requests

response = requests.get(
    "https://api.junis.ai/api/external/rag/datastores",
    headers={"X-API-Key": "jns_live_YOUR_API_KEY_HERE"}
)
print(response.json())

Response

Status Code: 200 OK

{
  "datastores": [
    {
      "datastore_id": "company-policies-a1b2c3d4",
      "display_name": "Company Policies",
      "description": "HR policies and employee handbook",
      "document_count": 5,
      "created_at": "2026-01-01T10:00:00Z",
      "is_active": true
    },
    {
      "datastore_id": "product-docs-e5f6g7h8",
      "display_name": "Product Documentation",
      "description": null,
      "document_count": 12,
      "created_at": "2026-01-01T08:30:00Z",
      "is_active": true
    }
  ],
  "total": 2
}

Response Fields

Field	Type	Description
`datastores`	array	List of DataStores
`datastores[].datastore_id`	string	Unique DataStore ID
`datastores[].display_name`	string	Human-readable name
`datastores[].description`	string	DataStore description (null if not set)
`datastores[].document_count`	integer	Number of completed uploads
`datastores[].created_at`	string	Creation timestamp (ISO 8601)
`datastores[].is_active`	boolean	Whether DataStore is active
`total`	integer	Total number of DataStores

List Uploads

Retrieve a paginated list of file uploads for your organization.

Endpoint

GET https://api.junis.ai/api/external/rag/uploads

Query Parameters

Parameter	Type	Required	Description
`limit`	integer	No	Maximum items to return (default: `50`, max: `100`)
`offset`	integer	No	Number of items to skip (default: `0`)
`status`	string	No	Filter by status: `pending`, `processing`, `indexing`, `completed`, `failed`
`datastore_id`	string	No	Filter by DataStore ID

Request Example

cURL

curl -X GET "https://api.junis.ai/api/external/rag/uploads?limit=10&status=completed" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE"

Python

import requests

response = requests.get(
    "https://api.junis.ai/api/external/rag/uploads",
    headers={"X-API-Key": "jns_live_YOUR_API_KEY_HERE"},
    params={
        "limit": 10,
        "status": "completed"
    }
)
print(response.json())

Response

Status Code: 200 OK

{
  "uploads": [
    {
      "upload_id": "550e8400-e29b-41d4-a716-446655440000",
      "filename": "policy.pdf",
      "file_size": 102400,
      "file_type": "application/pdf",
      "datastore_id": "company-policies-a1b2c3d4",
      "datastore_name": "Company Policies",
      "status": "completed",
      "uploaded_at": "2026-01-01T10:00:00Z",
      "indexed_at": "2026-01-01T10:05:00Z",
      "error_message": null
    }
  ],
  "total": 42,
  "limit": 10,
  "offset": 0
}

Upload Status Values

Status	Description
`pending`	Upload received, waiting for processing
`processing`	File being uploaded to GCS
`indexing`	File being indexed in Vertex AI Search
`completed`	Indexing complete, document is searchable
`failed`	Upload or indexing failed

Response Fields

Field	Type	Description
`uploads`	array	List of uploads
`uploads[].upload_id`	string	Unique upload ID (UUID)
`uploads[].filename`	string	Original filename
`uploads[].file_size`	integer	File size in bytes
`uploads[].file_type`	string	MIME type
`uploads[].datastore_id`	string	Target DataStore ID
`uploads[].datastore_name`	string	Target DataStore name
`uploads[].status`	string	Current status
`uploads[].uploaded_at`	string	Upload timestamp (ISO 8601)
`uploads[].indexed_at`	string	Indexing completion timestamp (null if not indexed)
`uploads[].error_message`	string	Error description if failed
`total`	integer	Total matching uploads
`limit`	integer	Items per page
`offset`	integer	Current offset

Get Upload Status

Retrieve detailed status of a specific upload.

Endpoint

GET https://api.junis.ai/api/external/rag/uploads/{upload_id}

Path Parameters

Parameter	Type	Required	Description
`upload_id`	string	Yes	Upload ID (UUID format)

Request Example

cURL

curl -X GET "https://api.junis.ai/api/external/rag/uploads/550e8400-e29b-41d4-a716-446655440000" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE"

Response

Status Code: 200 OK

{
  "upload_id": "550e8400-e29b-41d4-a716-446655440000",
  "filename": "policy.pdf",
  "file_size": 102400,
  "file_type": "application/pdf",
  "status": "completed",
  "ready": true,
  "datastore_id": "company-policies-a1b2c3d4",
  "datastore_name": "Company Policies",
  "uploaded_at": "2026-01-01T10:00:00Z",
  "indexed_at": "2026-01-01T10:05:00Z",
  "error_message": null
}

Response Fields

Field	Type	Description
`upload_id`	string	Unique upload ID (UUID)
`filename`	string	Original filename
`file_size`	integer	File size in bytes
`file_type`	string	MIME type
`status`	string	Current status
`ready`	boolean	`true` if document is searchable (`status === completed`)
`datastore_id`	string	Target DataStore ID
`datastore_name`	string	Target DataStore name
`uploaded_at`	string	Upload timestamp (ISO 8601)
`indexed_at`	string	Indexing completion timestamp (null if not indexed)
`error_message`	string	Error description if failed

Error Responses

404 Not Found

{
  "detail": {
    "error": {
      "message": "Upload not found: 550e8400-e29b-41d4-a716-446655440000",
      "type": "not_found",
      "code": "upload_not_found"
    }
  }
}

List DataStore Files

Retrieve all files in a specific DataStore with their indexing status.

Endpoint

GET https://api.junis.ai/api/external/rag/datastores/{datastore_id}/files

Path Parameters

Parameter	Type	Required	Description
`datastore_id`	string	Yes	DataStore ID

Request Example

cURL

curl -X GET "https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4/files" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE"

Python

import requests

response = requests.get(
    "https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4/files",
    headers={"X-API-Key": "jns_live_YOUR_API_KEY_HERE"}
)
print(response.json())

JavaScript

const response = await fetch(
  'https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4/files',
  {
    headers: { 'X-API-Key': 'jns_live_YOUR_API_KEY_HERE' }
  }
);
const data = await response.json();
console.log(data);

Response

Status Code: 200 OK

{
  "datastore_id": "company-policies-a1b2c3d4",
  "datastore_name": "Company Policies",
  "files": [
    {
      "name": "employee-handbook.pdf",
      "size": 2048576,
      "content_type": "application/pdf",
      "uploaded_at": "2026-01-01T10:00:00Z",
      "status": "completed"
    },
    {
      "name": "vacation-policy.docx",
      "size": 102400,
      "content_type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
      "uploaded_at": "2026-01-01T10:05:00Z",
      "status": "indexing"
    }
  ],
  "total_files": 2,
  "indexed_files": 1
}

Response Fields

Field	Type	Description
`datastore_id`	string	DataStore ID
`datastore_name`	string	DataStore display name
`files`	array	List of files in the DataStore
`files[].name`	string	Filename
`files[].size`	integer	File size in bytes
`files[].content_type`	string	MIME type
`files[].uploaded_at`	string	Upload timestamp (ISO 8601)
`files[].status`	string	Indexing status (`pending`, `indexing`, `completed`, `failed`)
`total_files`	integer	Total number of files
`indexed_files`	integer	Number of fully indexed files

Error Responses

404 Not Found

{
  "detail": {
    "error": {
      "message": "DataStore not found: invalid-datastore-id",
      "type": "not_found",
      "code": "datastore_not_found"
    }
  }
}

Delete DataStore

Delete a DataStore and all its contents (files and indexed documents).

Endpoint

DELETE https://api.junis.ai/api/external/rag/datastores/{datastore_id}

Destructive Operation: This permanently deletes all files and indexed documents in the DataStore. This action cannot be undone.

Path Parameters

Parameter	Type	Required	Description
`datastore_id`	string	Yes	DataStore ID to delete

Query Parameters

Parameter	Type	Required	Description
`force`	boolean	No	If `true`, delete even if DataStore is connected to agents (default: `false`)

Request Example

cURL

curl -X DELETE "https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4?force=true" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE"

Python

import requests

response = requests.delete(
    "https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4",
    headers={"X-API-Key": "jns_live_YOUR_API_KEY_HERE"},
    params={"force": True}
)
print(response.json())

JavaScript

const response = await fetch(
  'https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4?force=true',
  {
    method: 'DELETE',
    headers: { 'X-API-Key': 'jns_live_YOUR_API_KEY_HERE' }
  }
);
const data = await response.json();
console.log(data);

Response

Status Code: 200 OK

{
  "success": true,
  "datastore_id": "company-policies-a1b2c3d4",
  "message": "DataStore 'Company Policies' deleted successfully"
}

Response Fields

Field	Type	Description
`success`	boolean	Whether deletion was successful
`datastore_id`	string	Deleted DataStore ID
`message`	string	Success message

Error Responses

404 Not Found

{
  "detail": {
    "error": {
      "message": "DataStore not found: invalid-datastore-id",
      "type": "not_found",
      "code": "datastore_not_found"
    }
  }
}

409 Conflict - DataStore In Use

{
  "detail": {
    "error": {
      "message": "DataStore is connected to agents. Use force=true to delete anyway.",
      "type": "conflict",
      "code": "datastore_in_use",
      "connected_agents": ["brand-analyzer", "content-writer"]
    }
  }
}

Delete File

Delete a single file from a DataStore while preserving the DataStore itself.

Endpoint

DELETE https://api.junis.ai/api/external/rag/datastores/{datastore_id}/files/{filename}

Partial Deletion: Only the specified file is removed. The DataStore and other files remain intact.

Path Parameters

Parameter	Type	Required	Description
`datastore_id`	string	Yes	DataStore ID
`filename`	string	Yes	Name of the file to delete (URL-encoded if contains special characters)

Request Example

cURL

curl -X DELETE "https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4/files/old-policy.pdf" \
  -H "X-API-Key: jns_live_YOUR_API_KEY_HERE"

Python

import requests

response = requests.delete(
    "https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4/files/old-policy.pdf",
    headers={"X-API-Key": "jns_live_YOUR_API_KEY_HERE"}
)
print(response.json())

JavaScript

const response = await fetch(
  'https://api.junis.ai/api/external/rag/datastores/company-policies-a1b2c3d4/files/old-policy.pdf',
  {
    method: 'DELETE',
    headers: { 'X-API-Key': 'jns_live_YOUR_API_KEY_HERE' }
  }
);
const data = await response.json();
console.log(data);

Response

Status Code: 200 OK

{
  "success": true,
  "datastore_id": "company-policies-a1b2c3d4",
  "filename": "old-policy.pdf",
  "remaining_files": 4,
  "message": "File 'old-policy.pdf' deleted from 'Company Policies'"
}

Response Fields

Field	Type	Description
`success`	boolean	Whether deletion was successful
`datastore_id`	string	DataStore ID
`filename`	string	Deleted filename
`remaining_files`	integer	Number of files remaining in the DataStore
`message`	string	Success message

Error Responses

404 Not Found - DataStore

{
  "detail": {
    "error": {
      "message": "DataStore not found: invalid-datastore-id",
      "type": "not_found",
      "code": "datastore_not_found"
    }
  }
}

500 Internal Server Error - File Not Found

{
  "detail": {
    "error": {
      "message": "Failed to delete file: File not found in DataStore",
      "type": "deletion_error",
      "code": "file_deletion_failed"
    }
  }
}

Best Practices

Polling for Upload Completion

When an upload is in processing or indexing status, poll the status endpoint to check for completion:

import time
import requests

def wait_for_upload(upload_id, api_key, timeout=300, interval=5):
    """Poll until upload is completed or failed."""
    start = time.time()
    while time.time() - start < timeout:
        response = requests.get(
            f"https://api.junis.ai/api/external/rag/uploads/{upload_id}",
            headers={"X-API-Key": api_key}
        )
        data = response.json()

        if data["ready"]:
            return data
        elif data["status"] == "failed":
            raise Exception(f"Upload failed: {data.get('error_message')}")

        time.sleep(interval)

    raise TimeoutError("Upload indexing timed out")

Batch File Uploads

Upload multiple related files in a single request (up to 10 files):

import base64
import requests
import os

def upload_directory(datastore_id, directory_path, api_key):
    """Upload all supported files from a directory."""
    files = []
    for filename in os.listdir(directory_path):
        filepath = os.path.join(directory_path, filename)
        if os.path.isfile(filepath):
            with open(filepath, "rb") as f:
                content = base64.b64encode(f.read()).decode()
            files.append({
                "filename": filename,
                "content": content
            })

    response = requests.post(
        "https://api.junis.ai/api/external/rag/upload",
        headers={"X-API-Key": api_key, "Content-Type": "application/json"},
        json={"datastore_id": datastore_id, "files": files[:10]}  # Max 10
    )
    return response.json()

Organizing DataStores

Create separate DataStores for different topics or document types:

By Department: “HR Policies”, “Engineering Docs”, “Sales Materials”
By Project: “Project Alpha”, “Project Beta”
By Type: “Legal Contracts”, “Technical Specs”, “Meeting Notes”

This helps agents retrieve more relevant information by searching specific DataStores.

File Size Optimization

For large documents:

Split large PDFs into smaller chapters
Remove unnecessary images from documents
Use text formats (TXT, MD) when formatting isn’t important
Compress images before including in documents

Authentication

Learn about API key scopes and authentication