RAG

#Overview
The RAG (Retrieval-Augmented Generation) feature allows users to upload and manage documents that can later be used by AI agents for accurate, context-aware responses. This feature helps the system retrieve relevant information from user-provided documents instead of relying only on a base model.
#What Can You Do With RAG?
Upload different types of documents (PDF, text, URL-based content, content etc.)
Control how documents are split (chunked) for better retrieval
Choose performance vs accuracy based on use case
Store documents in specific collections for targeted querying
#Create Knowledge Base 
#1. Name (Required)
Enter a meaningful name for the knowledge base.
Example: Demo PDF
This name helps you identify the document collection later.
#2. Description (Required)
Add a short description explaining what the document contains.
Example: This is demo pdf.
This is useful for clarity when managing multiple knowledge bases.
#3. Upload Method (Required)
Users can choose one of the following methods:
#a. URL
Provide a public URL
The system will fetch and process the content automatically
#b. Upload File (Selected in UI)
Upload files such as Images, PDFs, and TXT files.
#c. Content
Paste raw text directly into the system
Useful for quick notes or small content
#5. Chunking Type
Defines how the document is split into smaller parts (chunks) for retrieval.
Users can choose from multiple chunking strategies based on their document type and use case:
#a. Agentic
Uses an AI-driven (agent-based) approach to chunk documents
Chunks are created based on how an AI agent would reason over the content
#b. Recursive
Intelligently splits content by sections, paragraphs, and sentences
Recommended for structured documents like PDFs, manuals, and reports
#c. Semantic
Splits content based on semantic meaning rather than fixed structure
Best for unstructured or narrative content
#6. Chunk Size and Chunk Overlap 
#Chunk Size
Maximum size of each semantic chunk (e.g., 4000 characters)
Larger values preserve more context but may impact retrieval speed
#Chunk Overlap
Number of overlapping characters between consecutive chunks (e.g., 200)
Ensures important context is not lost at chunk boundaries
Recommended for long paragraphs or flowing narrative content
#Advanced Settings
#Query Access Type
Allows users to optimize between speed and accuracy:
Fastest: Lowest latency and best for quick responses.
Moderate: Balanced speed and accuracy.
High Accuracy: More processing and best for complex or critical queries.
#APIs Involved
#1. Upload API
Upload different types of documents (Images, PDFs, and TXT files.)
curl -X POST 'https://api.gtwy.ai/image/processing/upload' \
  -H 'pauthkey: <YOUR_GENERATED_PAUTHKEY>' \
  -F 'file=@<FILE_PATH>' 
API Response:
{  
"success": true,  
"file_url": "https://resources.gtwy.ai/uploads/sample.pdf"
}
#2. Resource Create API
Used to add the uploaded document into a specific collection.
Collection selection depends on the user’s Advanced Settings.
Resource Types:
URL Resource: You can provide your own URL in the Resource API.
File Upload Resource:
Used when a file is uploaded via the Upload API, the url must be the file_url returned by the Upload API.
Custom URLs are not allowed in this case. 
Content (Text) Resource
Used when you want to upload raw text/content.
Include the content key in the Resource API request.
{
    "content": "my content"
}
curl -X POST 'https://db.gtwy.ai/api/rag/resource' \
  -H 'pauthkey: <YOUR_GENERATED_PAUTHKEY>' \
  -H 'Content-Type: application/json' \
  -d '{
    "title": "Demo Link",
    "description": "Description of doc",
    "settings": {
      "strategy": "recursive",
      "chunkSize": 4000
    },
    "url": "https://demo.com/",
    "collection_details": "fastest",
    "owner_id": "Your owner_id" //optional
  }'
API Response: 
{...}{
    "success": true,
    "message": "Resource created successfully",
    "data": {
        "title": "AI AGENTS",
        "collectionId": "6968d081f406ec4f14f8e3f3",
        "ownerId": "gtwy",
        "description": "An AI Agents document explains how AI systems work on their own to understand tasks, make decisions, and take actions.",
        "url": " https://resources.gtwy.ai/uploads/sample.pdf",
        "refreshedAt": null,
        "isDeleted": false,
        "settings": {
            "chunkSize": 4000,
            "strategy": "recursive"
        },
        "_id": "698592e4c0f1c813e39eeb54",
        "createdAt": "2026-02-06T07:06:12.414Z",
        "updatedAt": "2026-02-06T07:06:12.414Z"
    }
}

Key Fields Explained:
Field
Description
owner_id
Unique identifier of the document owner
collection_details
Retrieval preference (fastest, moderate, high_accuracy)
settings.strategy
Chunking type used during ingestion

#3. Search/Query in Document API
This cURL is used for searching in RAG.
Use query to pass the search/query text.
Use doc_id to search within a specific document only.
For collection-level search, the owner_id must be saved when the document is added and provided again during search.
If owner_id is missing, the search will not work and no response will be returned.
When searching by doc_id, collection_id and owner_id are not required.
Api for searching on document: 
curl -X POST 'https://api.gtwy.ai/rag/query' \  
-H 'pauthkey: <YOUR_GENERATED_PAUTHKEY>' \  
-H 'Content-Type: application/json' \  
-d '{    
"query" : "Your Query?",
"doc_id" : "Your document ID or resource ID " 
}'
API for searching on collection:
curl -X POST 'https://api.gtwy.ai/rag/query' \  
-H 'pauthkey: <YOUR_GENERATED_PAUTHKEY>' \  
-H 'Content-Type: application/json' \  
-d '{    
"collection_id": "<COLLECTION_ID>",   
"owner_id": "your owner ID",  
"query": "testing",
"score": 0.3 //match response accuracy with score
}'
📌Note: If an owner_id is provided while creating a resource, the same owner_id must also be passed when performing a search/query.
and score is to return only highly relevant chunks based on user query.

API Response: 
{
    "success": true,
    "text": "Response text",
    "results": [
        {
            "id": "369fce38-ed3e-fcce-97b4-51094ea35dcd",
            "version": 153,
            "score": 0.83372265,
            "payload": {
                "resourceId": "6985aeb6d923b3729e665c3e",
                "collectionId": "6968d081f406ec4f14f8e3f3",
                "content": "CHUNK TEXT 1",
                "ownerId": "gtwy"
            }
        },
        {
            "id": "6d2cbf32-0131-8837-e1fe-87e367c6c310",
            "version": 147,
            "score": 0.8082152,
            "payload": {
                "resourceId": "6985adc61f5b4176d9fd033b",
                "collectionId": "6968d081f406ec4f14f8e3f3",
                "content": "CHUNK TEXT 2",
                "ownerId": "gtwy"
            }
        },
        {
            "id": "b5947b57-e66c-1b63-d4e6-fcdfcd016944",
            "version": 149,
            "score": 0.76357174,
            "payload": {
                "resourceId": "698592e4c0f1c813e39eeb54",
                "collectionId": "6968d081f406ec4f14f8e3f3",
                "content": "CHUNK TEXT 3",
                "ownerId": "gtwy"
            }
        },
        {
            "id": "936bc918-664a-1a85-4cb7-2f6fd38ca851",
            "version": 153,
            "score": 0.7241006,
            "payload": {
                "resourceId": "6985aeb6d923b3729e665c3e",
                "collectionId": "6968d081f406ec4f14f8e3f3",
                "content": "CHUNK TEXT 4",
                "ownerId": "gtwy
            }
        },
        {
            "id": "dce77c6b-a16e-6429-9755-b126173763f2",
            "version": 153,
            "score": 0.7064505,
            "payload": {
                "resourceId": "6985aeb6d923b3729e665c3e",
                "collectionId": "6968d081f406ec4f14f8e3f3",
                "content": "CHUNK TEXT 5",
                "ownerId": "gtwy"
            }
        }
    ]
}

#4. Update Document API
The Update Document API is used to modify an existing document based on its document type.
Content-based documents
You can update: title, description, content
This allows you to fully update the document content and details.
File or Link-based documents
You can update: title, description, url
The original uploaded file or link remains unchanged
To update a file-based document, you must first upload the new file and use the newly generated file URL in the update request.
For link-based documents, you can directly provide the updated URL.
curl -X PUT 'https://db.gtwy.ai/api/rag/resource/{{doc_id}}' \  
-H 'pauthkey: <YOUR_GENERATED_PAUTHKEY>' \  
-H 'Content-Type: application/json' \  
-d '{   
"title": "Updated Document Title",    
"description": "Updated Description of document",     
"content": "Updated Doc Content"  
}'
📌Note: To update a File and URL-based document, include the url field in the API request body instead of the content field.

API Response: 
{...}{
    "success": true,
    "message": "Resource updated successfully",
    "data": {
        "_id": "6985aeb6d923b3729e665c3e",
        "title": "AI AGENTS",
        "collectionId": "6968d081f406ec4f14f8e3f3",
        "ownerId": "gtwy",
        "description": "An AI Agents document explains how AI systems work on their own to understand tasks, make decisions, and take actions.",
        "url": "https://resources.gtwy.ai/uploads/sample.pdf",
        "refreshedAt": "2026-02-06T09:21:55.491Z",
        "isDeleted": false,
        "settings": {
            "chunkSize": 500,
            "strategy": "recursive"
        },
        "createdAt": "2026-02-06T09:04:54.341Z",
        "updatedAt": "2026-02-06T09:21:55.491Z",
        "content": "UPDATED CONTENT",
        "metadata": {
            "status": "chunked"
        }
    }
}

#5. Delete Document API
The Delete Document API is used to delete a document from the RAG system using its document ID.
curl -X DELETE 'https://db.gtwy.ai/api/rag/resource/{{doc_id}}' \  
-H 'pauthkey: <YOUR_GENERATED_PAUTHKEY>' \  
-H 'Accept: application/json'
API Response:
{...}{
    "success": true,
    "message": "Resource deleted successfully",
    "data": {
        "_id": "6985b4691f5b4176d9fd0355",
        "title": "AI AGENTS",
        "collectionId": "6968d081f406ec4f14f8e3f3",
        "ownerId": "gtwy",
        "description": "An AI Agents document explains how AI systems work on their own to understand tasks, make decisions, and take actions.",
        "url": "https://resources.gtwy.ai/uploads/sample.pdf",
        "refreshedAt": null,
        "isDeleted": true,
        "settings": {
            "chunkSize": 500,
            "strategy": "recursive"
        },
        "createdAt": "2026-02-06T09:29:13.065Z",
        "updatedAt": "2026-02-06T09:29:38.082Z",
        "content": "DOC CONTENT",
        "metadata": {
            "status": "chunked"
        }
    }
}

#6. Get All Documents/Resources API
This API retrieves all RAG resources associated with a specific collection. The collectionId and ownerId are passed as query parameters to fetch relevant resources for the given user and collection.
curl --location --request GET \
'https://db.gtwy.ai/api/rag/resource/by-collection?collectionId={{COLLECTION_ID}}&ownerId={{OWNER_ID}}' \
--header 'Content-Type: application/json' \
--header 'pauthkey: <YOUR_GENERATED_PAUTHKEY>''
Key Fields Explained:
Parameter
Type
Description
collectionId
Query
Unique ID of the RAG collection
ownerId
Query
Owner/User identifier ID
pauthkey
Header
Authentication key
📌Both collectionId and ownerId are required parameters.

API Response:
{...}{
    "success": true,
    "message": "Resources fetched for the specified collection",
    "data": {
        "resources": [
            {
                "_id": "698ae1091f5b4176d9fd0cd9",
                "title": "AIML",
                "collectionId": "6966583929cff36bc9f9d030",
                "ownerId": "gtwy",
                "description": "Artificial Intelligence, Machine Learning and AI Agents",
                "url": "https://resources.gtwy.ai/uploads/a0dc2297-f3a0-4107-bd2f-af87fc873217.pdf",
                "refreshedAt": null,
                "isDeleted": false,
                "settings": {
                    "chunkSize": 4000,
                    "chunkOverlap": 200,
                    "strategy": "semantic"
                },
                "createdAt": "2026-02-10T07:40:57.881Z",
                "updatedAt": "2026-02-10T07:40:58.353Z",
                "content": "AI, Machine Learning, and AI Agents – Overview\n1. Artificial Intelligence (AI)\nArtificial Intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to\nthink, learn, and make decisions.",
                "metadata": {
                    "status": "chunked"
                }
            },
            {
                "_id": "698ae1631f5b4176d9fd0cdb",
                "title": "AIML",
                "collectionId": "6966583929cff36bc9f9d030",
                "ownerId": "gtwy",
                "description": "Artificial Intelligence, Machine Learning and AI Agents",
                "url": "https://resources.gtwy.ai/uploads/a5dedb91-89eb-4896-88e4-a54b3b7b8aff.pdf",
                "refreshedAt": null,
                "isDeleted": false,
                "settings": {
                    "chunkSize": 4000,
                    "chunkOverlap": 200,
                    "strategy": "semantic"
                },
                "createdAt": "2026-02-10T07:42:27.207Z",
                "updatedAt": "2026-02-10T07:42:27.671Z",
                "content": "Disadvantages of Artificial Intelligence (AI)\nThe main disadvantages of AI include job displacement, high implementation costs, bias and\ndiscrimination from skewed data, privacy and security risks, and a lack of human creativity,\nempathy, and emotional intelligence.",
                "metadata": {
                    "status": "chunked"
                }
            },
            {
                "_id": "698ae19dd923b3729e666588",
                "title": "Google",
                "collectionId": "6966583929cff36bc9f9d030",
                "ownerId": "gtwy",
                "description": "Google Description",
                "url": "https://google.com/",
                "refreshedAt": null,
                "isDeleted": false,
                "settings": {
                    "chunkSize": 4000,
                    "chunkOverlap": 200,
                    "strategy": "semantic"
                },
                "createdAt": "2026-02-10T07:43:25.403Z",
                "updatedAt": "2026-02-10T07:43:29.608Z",
                "content": "About\nGoogle\nDoc",
                "metadata": {
                    "status": "chunked"
                }
            }
        ],
        "created": 0
    }
}

Field	Description
`owner_id`	Unique identifier of the document owner
`collection_details`	Retrieval preference (`fastest`, `moderate`, `high_accuracy`)
`settings.strategy`	Chunking type used during ingestion

Parameter	Type	Description
`collectionId`	Query	Unique ID of the RAG collection
`ownerId`	Query	Owner/User identifier ID
`pauthkey`	Header	Authentication key

Was this helpful?

Built with DocStar

RAG

Overview

What Can You Do With RAG?

Create Knowledge Base

1. Name (Required)

2. Description (Required)

3. Upload Method (Required)

a. URL

b. Upload File (Selected in UI)

c. Content

5. Chunking Type

a. Agentic

b. Recursive

c. Semantic

6. Chunk Size and Chunk Overlap

Chunk Size

Chunk Overlap

Advanced Settings

Query Access Type

APIs Involved

1. Upload API

2. Resource Create API

3. Search/Query in Document API

4. Update Document API

5. Delete Document API

6. Get All Documents/Resources API

On this page