BigScale API Documentation

Overview

BigScale enables you to ingest AI conversations, automatically extract user signals (interests, intent, preferences), store memories, and query rich user profiles—all in real-time at scale.

How it works

Three simple steps from raw conversations to actionable intelligence:

Step 01

Ingest

Send messages as they happen. Single or batch.

Step 02

Extract

AI extracts interests, intent, preferences, and memories.

Step 03

Query

Get signals, profiles, or summaries at any token length.

Key Features

⚡

Real-time Signals

Extract interests, current intent, and preferences from every message as it happens. No batch processing needed.

↔

Flexible Summaries

Request summaries at any token length—from 10-token taglines to 500k full context windows. One API, infinite flexibility.

◫

Scoped Memory

Query by user, conversation, or last K turns. Control exactly what context you need for each use case.

Quick Start

Get started in two API calls: ingest a conversation, then query the user.

1. Ingest a conversation

# Ingest messages from a conversation
curl -X POST https://api.configure.dev/v1/bigscale/ingest \
  -H "Content-Type: application/json" \
  -H "X-App-Key: your-api-key" \
  -d '{
    "userId": "user_123",
    "conversationId": "conv_456",
    "messages": [
      { "role": "user", "content": "I need running shoes under $100" },
      { "role": "assistant", "content": "I can help with that!" }
    ]
  }'
                

2. Query user signals

# Get extracted signals for the user
curl "https://api.configure.dev/v1/bigscale/user/user_123/signals" \
  -H "X-App-Key: your-api-key"

# Response
{
  "success": true,
  "signals": {
    "interests": ["running shoes"],
    "currentIntent": "purchase running shoes",
    "preferences": { "budget": "under $100" }
  }
}
                

Ingest Endpoints

POST /ingest

Ingest a single conversation with one or more messages. Signals are extracted from user messages in real-time. Use this for streaming or real-time ingestion.

Request Body

Parameter	Type	Description
userId required	string	Your unique user identifier
conversationId required	string	Conversation/session identifier
messages required	array	Array of `{ role: "user"\|"assistant", content: string }`

Response

{
  "success": true,
  "conversationId": "conv_456",
  "messagesProcessed": 2,
  "signalsUpdated": true,
  "memoriesCreated": 1,
  "extractedSignals": {
    "interests": ["running shoes"],
    "intent": "purchase",
    "preferences": { "budget": "under $100" }
  }
}
                        

POST /ingest/batch

Bulk ingest hundreds or thousands of conversations at once. By default runs async and returns a job ID to poll for completion. Extracts signals, memories, and generates titles in parallel.

Request Body

Parameter	Type	Description
conversations required	array	Array of `{ userId, conversationId, messages[] }`
sync	boolean	If true, waits for completion (default: false)

Response (Async)

{
  "success": true,
  "jobId": "ingest_1234567890_abc123",
  "status": "processing",
  "messagesQueued": 1500
}
                        

GET /ingest/job/:jobId

Poll for batch ingest job status. Returns progress and results when complete.

Path Parameters

Parameter	Type	Description
jobId required	string	Job ID returned from batch ingest

Response

{
  "success": true,
  "jobId": "ingest_1234567890_abc123",
  "status": "completed",  // processing | completed | failed
  "duration": 2340,
  "messagesProcessed": 1500,
  "signalsExtracted": 450,
  "memoriesCreated": 127
}
                        

Query Endpoints

GET /user/:userId

Get user signals and/or an AI-generated summary at any token length. The summary is generated on-demand based on the user's conversation history and memories. Use for personalization, ad targeting, or context injection.

Query Parameters

Parameter	Type	Description
maxTokens	number	Summary length (10 to 500,000). Default: 500
scope	string	`user` \| `conversation:id` \| `last_k_turns:N`
signals	boolean	Include structured signals. Default: true
summary	boolean	Include text summary. Default: true

Response

{
  "success": true,
  "signals": {
    "interests": ["running", "fitness", "travel"],
    "currentIntent": "looking for running shoes",
    "preferences": { "budget": "moderate" },
    "recentTopics": ["half marathon", "japan trip"]
  },
  "summary": "Maya is a 26-year-old product manager interested in fitness and travel...",
  "summaryTokens": 487
}
                        

GET /user/:userId/signals

Fast signals-only query. Returns cached structured signals without generating a summary. Use when you only need interests, intent, and preferences—much faster than the full query.

Response

{
  "success": true,
  "signals": {
    "interests": ["running", "fitness"],
    "currentIntent": "purchase running shoes",
    "preferences": { "budget": "under $100" },
    "recentTopics": ["marathon training"],
    "updatedAt": "2025-01-15T10:30:00Z"
  }
}
                        

POST /users/batch

Batch query for multiple users—get signals and summaries for many users at once. Ideal for batch ad targeting or bulk personalization. Returns a job ID for async processing.

Request Body

Parameter	Type	Description
userIds required	string[]	Array of user IDs to query
maxTokens	number	Summary length for all users
scope	string	Scope to apply for all users

Profile Endpoints

GET /profile/:userId

Get a clean user profile optimized for ad targeting. Returns structured interests, intent, preferences, and recent topics without generating a summary.

Response

{
  "success": true,
  "profile": {
    "userId": "user_123",
    "interests": ["travel", "fitness", "japan"],
    "currentIntent": "looking for budget running shoes",
    "preferences": { "budget": "under $100" },
    "recentTopics": ["half marathon", "japan trip"],
    "conversationCount": 15,
    "lastActive": "2025-01-15T10:30:00Z"
  }
}
                        

POST /profiles/batch

Get profiles for multiple users in one request. Synchronous and fast—limited to 100 users per request. Perfect for real-time batch lookups.

Request Body

Parameter	Type	Description
userIds required	string[]	Array of user IDs (max 100)

Response

{
  "success": true,
  "count": 3,
  "profiles": {
    "user_123": { "interests": [...], ... },
    "user_456": { "interests": [...], ... },
    "user_789": null  // not found
  }
}
                        

GET /user/:userId/profile/full

Get a comprehensive rich profile derived from all memories and signals. Includes AI-generated bio, personality traits, goals, interests, memory highlights, and recent activity summary.

Query Parameters

Parameter	Type	Description
refresh	boolean	Force regeneration of profile (default: false)

Response

{
  "success": true,
  "profile": {
    "bio": "Maya is a 26-year-old product manager...",
    "traits": ["curious", "health-conscious", "adventurous"],
    "currentGoals": ["run a half marathon", "visit Japan"],
    "interests": ["running", "travel", "cooking"],
    "memoryHighlights": ["Training for first half marathon", ...],
    "recentActivity": "Discussed running gear and Japan travel plans...",
    "stats": { "memoryCount": 24, "conversationCount": 15 }
  }
}
                        

Memory Endpoints

GET /user/:userId/memories

Get all extracted memories for a user. Memories are discrete facts, preferences, or experiences extracted from conversations. Can filter by category.

Query Parameters

Parameter	Type	Description
category	string	Filter by memory category (e.g., "preference", "fact", "goal")

Response

{
  "success": true,
  "count": 24,
  "memories": [
    {
      "id": "mem_abc123",
      "content": "Prefers Nike or Adidas running shoes",
      "category": "preference",
      "sourceConversationId": "conv_456",
      "createdAt": 1705312200000
    },
    ...
  ]
}
                        

GET /conversation/:conversationId/memories

Get memories extracted from a specific conversation. Useful for reviewing what was learned from a particular interaction.

DELETE /user/:userId/memories/:memoryId

Delete a specific memory. Use for user-requested data deletion or memory correction.

Configuration

GET /config/memory-scope

Get the current memory scope configuration. The memory scope defines what information gets extracted as memories during conversation ingest.

PUT /config/memory-scope

Update the memory scope configuration. Customize the extraction prompt and categories to match your use case.

Request Body

Parameter	Type	Description
name	string	Scope name identifier
prompt required	string	The extraction prompt
categories	string[]	Memory categories to extract

POST /config/memory-scope/reset

Reset memory scope to the default configuration.

Utility Endpoints

GET /stats

Get system statistics including total conversations, messages, and memories.

Response

{
  "success": true,
  "totalConversations": 1250,
  "totalMessages": 15600,
  "totalMemories": 3420
}
                        

GET /user/:userId/conversations

Get all conversations for a user. Returns conversation metadata including title, topic, mood, and message count.

GET /conversation/:conversationId

Get a full conversation with all messages.

GET /job/:jobId

Check any batch job status (ingest or query batch jobs). Alias for /ingest/job/:jobId.

GET /sample-data

Get sample conversations for testing batch ingest. Returns pre-formatted data ready for the /ingest/batch endpoint.