FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(web): rename PostHog chat events, add tool_used tracking, and client-side ask events by msukkari · Pull Request #1111 · sourcebot-dev/sourcebot · GitHub

Repository navigation

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension .md  (2) .ts  (11) .tsx  (1) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
1 change: 1 addition & 0 deletions CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- Deprecated `GOOGLE_VERTEX_THINKING_BUDGET_TOKENS` environment variable in favor of per-model `thinkingBudget` config. [#1110](https://github.com/sourcebot-dev/sourcebot/pull/1110)
- Removed `GOOGLE_VERTEX_INCLUDE_THOUGHTS` environment variable. Thoughts are now always included. [#1110](https://github.com/sourcebot-dev/sourcebot/pull/1110)
- Renamed and consolidated PostHog chat events (`wa_chat_thread_created` -> `ask_thread_created`, `wa_chat_message_sent` -> `ask_message_sent`, `wa_chat_tool_used` -> `tool_used`), added unified `tool_used` tracking across the ask agent and MCP server, and removed the redundant `api_code_search_request` event. [#1111](https://github.com/sourcebot-dev/sourcebot/pull/1111)

## [4.16.8] - 2026-04-09

Expand Down
6 changes: 6 additions & 0 deletions CLAUDE.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,12 @@ if (!value) { return; }
if (condition) doSomething();
```

## PostHog Event Naming

- The `wa_` prefix is reserved for events that can ONLY be fired from the web app (e.g., `wa_login_with_github`, `wa_chat_feedback_submitted`).
- Events fired from multiple sources (web app, MCP server, API) must NOT use the `wa_` prefix (e.g., `ask_message_sent`, `tool_used`).
- Multi-source events should include a `source` property to identify the origin (e.g., `'sourcebot-web-client'`, `'sourcebot-mcp-server'`, `'sourcebot-ask-agent'`).

## Tailwind CSS

Use Tailwind color classes directly instead of CSS variable syntax:
Expand Down
7 changes: 5 additions & 2 deletions packages/web/src/app/api/(server)/chat/route.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -91,12 +91,15 @@ export const POST = apiHandler(async (req: NextRequest) => {
return [];
}))).flat();

await captureEvent('wa_chat_message_sent', {
const source = req.headers.get('X-Sourcebot-Client-Source') ?? undefined;

await captureEvent('ask_message_sent', {
chatId: id,
messageCount: messages.length,
selectedReposCount: expandedRepos.length,
source,
...(env.EXPERIMENT_ASK_GH_ENABLED === 'true' ? { selectedRepos: expandedRepos } : {}),
} );
});
Comment thread
msukkari marked this conversation as resolved.
Resolved

const stream = await createMessageStream({
chatId: id,
Expand Down
7 changes: 0 additions & 7 deletions packages/web/src/app/api/(server)/search/route.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import { search, searchRequestSchema } from "@/features/search";
import { apiHandler } from "@/lib/apiHandler";
import { captureEvent } from "@/lib/posthog";
import { requestBodySchemaValidationError, serviceErrorResponse } from "@/lib/serviceError";
import { isServiceError } from "@/lib/utils";
import { NextRequest } from "next/server";
Expand All @@ -21,12 +20,6 @@ export const POST = apiHandler(async (request: NextRequest) => {
...options
} = parsed.data;

const source = request.headers.get('X-Sourcebot-Client-Source') ?? 'unknown';
await captureEvent('api_code_search_request', {
source,
type: 'blocking',
});

const response = await search({
queryType: 'string',
query,
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import { streamSearch, searchRequestSchema } from '@/features/search';
import { apiHandler } from '@/lib/apiHandler';
import { captureEvent } from '@/lib/posthog';
import { requestBodySchemaValidationError, serviceErrorResponse } from '@/lib/serviceError';
import { isServiceError } from '@/lib/utils';
import { NextRequest } from 'next/server';
Expand All @@ -20,12 +19,6 @@ export const POST = apiHandler(async (request: NextRequest) => {
...options
} = parsed.data;

const source = request.headers.get('X-Sourcebot-Client-Source') ?? 'unknown';
await captureEvent('api_code_search_request', {
source,
type: 'streamed',
});

const stream = await streamSearch({
queryType: 'string',
query,
Expand Down
3 changes: 2 additions & 1 deletion packages/web/src/features/chat/actions.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,10 @@ export const createChat = async ({ source }: { source?: string } = {}) => sew(()
});
}

await captureEvent('wa_chat_thread_created', {
await captureEvent('ask_thread_created', {
chatId: chat.id,
isAnonymous: isGuestUser,
source,
});
Comment thread
msukkari marked this conversation as resolved.
Resolved

return {
Expand Down
9 changes: 1 addition & 8 deletions packages/web/src/features/chat/agent.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import { SBChatMessage, SBChatMessageMetadata } from "@/features/chat/types";
import { getAnswerPartFromAssistantMessage } from "@/features/chat/utils";
import { getFileSource } from '@/features/git';
import { captureEvent } from "@/lib/posthog";
import { isServiceError } from "@/lib/utils";
import { LanguageModelV3 as AISDKLanguageModelV3 } from "@ai-sdk/provider";
import { ProviderOptions } from "@ai-sdk/provider-utils";
Expand Down Expand Up @@ -210,13 +209,7 @@ const createAgentStream = async ({
],
toolChoice: "auto",
onStepFinish: ({ toolResults }) => {
toolResults.forEach(({ toolName, output, dynamic }) => {
captureEvent('wa_chat_tool_used', {
chatId,
toolName,
success: !isServiceError(output),
});

toolResults.forEach(({ output, dynamic }) => {
if (dynamic || isServiceError(output)) {
return;
}
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ export const ChatThread = ({
messages: initialMessages,
transport: new DefaultChatTransport({
api: '/api/chat',
headers: {
'X-Sourcebot-Client-Source': 'sourcebot-web-client',
},
}),
onData: (dataPart) => {
// Keeps sources added by the assistant in sync.
Expand Down
6 changes: 4 additions & 2 deletions packages/web/src/features/mcp/askCodebase.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,10 @@ export const askCodebase = (params: AskCodebaseParams): Promise<AskCodebaseResul
},
});

await captureEvent('wa_chat_thread_created', {
await captureEvent('ask_thread_created', {
chatId: chat.id,
isAnonymous: !user,
source,
});
Comment thread
msukkari marked this conversation as resolved.
Resolved

if (user) {
Expand Down Expand Up @@ -137,10 +138,11 @@ export const askCodebase = (params: AskCodebaseParams): Promise<AskCodebaseResul

let finalMessages: SBChatMessage[] = [];

await captureEvent('wa_chat_message_sent', {
await captureEvent('ask_message_sent', {
chatId: chat.id,
messageCount: 1,
selectedReposCount: selectedRepos.length,
source,
...(env.EXPERIMENT_ASK_GH_ENABLED === 'true' ? {
selectedRepos: selectedRepos.map(r => r.value)
} : {}),
Expand Down
17 changes: 17 additions & 0 deletions packages/web/src/features/mcp/server.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import {
languageModelInfoSchema,
} from '@/features/chat/types';
import { askCodebase } from '@/features/mcp/askCodebase';
import { captureEvent } from '@/lib/posthog';
import { isServiceError } from '@/lib/utils';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { ChatVisibility } from '@sourcebot/db';
Expand Down Expand Up @@ -57,6 +58,11 @@ export async function createMcpServer(): Promise<McpServer> {
},
async () => {
const models = await getConfiguredLanguageModelsInfo();
captureEvent('tool_used', {
toolName: 'list_language_models',
source: 'sourcebot-mcp-server',
success: true,
});
return { content: [{ type: "text", text: JSON.stringify(models) }] };
}
);
Expand Down Expand Up @@ -101,11 +107,22 @@ export async function createMcpServer(): Promise<McpServer> {
});

if (isServiceError(result)) {
captureEvent('tool_used', {
toolName: 'ask_codebase',
source: 'sourcebot-mcp-server',
success: false,
});
return {
content: [{ type: "text", text: `Failed to ask codebase: ${result.message}` }],
};
}

captureEvent('tool_used', {
toolName: 'ask_codebase',
source: 'sourcebot-mcp-server',
success: true,
});

const formattedResponse = dedent`
${result.answer}

Expand Down
25 changes: 24 additions & 1 deletion packages/web/src/features/tools/adapters.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { tool } from "ai";
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";
import { captureEvent } from "@/lib/posthog";
import { ToolContext, ToolDefinition } from "./types";

export function toVercelAITool<TName extends string, TShape extends z.ZodRawShape, TMetadata>(
Expand All @@ -11,7 +12,21 @@ export function toVercelAITool<TName extends string, TShape extends z.ZodRawShap
description: def.description,
inputSchema: def.inputSchema,
title: def.title,
execute: (input) => def.execute(input, context),
execute: async (input) => {
let success = true;
try {
return await def.execute(input, context);
} catch (error) {
success = false;
throw error;
} finally {
captureEvent('tool_used', {
toolName: def.name,
source: context.source ?? 'unknown',
success,
});
}
},
toModelOutput: ({ output }) => ({
type: "content",
value: [{ type: "text", text: output.output }],
Expand All @@ -38,13 +53,21 @@ export function registerMcpTool<TName extends string, TShape extends z.ZodRawSha
},
},
async (input) => {
let success = true;
try {
const parsed = def.inputSchema.parse(input);
const result = await def.execute(parsed, context);
return { content: [{ type: "text" as const, text: result.output }] };
} catch (error) {
success = false;
const message = error instanceof Error ? error.message : String(error);
return { content: [{ type: "text" as const, text: `Tool "${def.name}" failed: ${message}` }], isError: true };
} finally {
captureEvent('tool_used', {
toolName: def.name,
source: context.source ?? 'unknown',
success,
});
}
},
);
Expand Down
4 changes: 1 addition & 3 deletions packages/web/src/lib/apiHandler.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,7 @@ export function apiHandler<H extends AnyHandler>(
const source = request.headers.get('X-Sourcebot-Client-Source') ?? 'unknown';

// Fire and forget - don't await to avoid blocking the request
captureEvent('api_request', { path, method, source }).catch(() => {
// Silently ignore tracking errors
});
captureEvent('api_request', { path, method, source });
}

// Call the original handler with all arguments
Expand Down
72 changes: 34 additions & 38 deletions packages/web/src/lib/posthog.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { PostHog } from 'posthog-node'
import { env, SOURCEBOT_VERSION } from '@sourcebot/shared'
import { createLogger, env, SOURCEBOT_VERSION } from '@sourcebot/shared'
import { RequestCookies } from 'next/dist/compiled/@edge-runtime/cookies';
import * as Sentry from "@sentry/nextjs";
import { PosthogEvent, PosthogEventMap } from './posthogEvents';
import { cookies, headers } from 'next/headers';
import { auth } from '@/auth';
import { getVerifiedApiObject } from '@/middleware/withAuth';
import { getAuthenticatedUser } from '@/middleware/withAuth';

const logger = createLogger('posthog');

/**
* @note: This is a subset of the properties stored in the
Expand Down Expand Up @@ -53,28 +54,19 @@ const getPostHogCookie = (cookieStore: Pick<RequestCookies, 'get'>): PostHogCook
* Attempts to retrieve the distinct id of the current user.
*/
export const tryGetPostHogDistinctId = async () => {
// First, attempt to retrieve the distinct id from the cookie.
// First, attempt to retrieve the distinct id from the PostHog cookie
// (set by the client-side PostHog SDK). This preserves identity
// continuity between client-side and server-side events.
const cookieStore = await cookies();
const cookie = getPostHogCookie(cookieStore);
if (cookie) {
return cookie.distinct_id;
}

// Next, from the session.
const session = await auth();
if (session) {
return session.user.id;
}

// Finally, from the api key.
const headersList = await headers();
const apiKeyString = headersList.get("X-Sourcebot-Api-Key") ?? undefined;
if (!apiKeyString) {
return undefined;
}

const apiKey = await getVerifiedApiObject(apiKeyString);
return apiKey?.createdById;
// Fall back to the authenticated user's ID. This covers all auth
// methods: session cookies, OAuth Bearer tokens, and API keys.
const authResult = await getAuthenticatedUser();
return authResult?.user.id;
}

export const createPostHogClient = async () => {
Expand All @@ -88,24 +80,28 @@ export const createPostHogClient = async () => {
}

export async function captureEvent<E extends PosthogEvent>(event: E, properties: PosthogEventMap[E]) {
if (env.SOURCEBOT_TELEMETRY_DISABLED === 'true') {
return;
}
try {
if (env.SOURCEBOT_TELEMETRY_DISABLED === 'true') {
return;
}

const distinctId = await tryGetPostHogDistinctId();
const posthog = await createPostHogClient();

const headersList = await headers();
const host = headersList.get("host") ?? undefined;

posthog.capture({
event,
properties: {
...properties,
sourcebot_version: SOURCEBOT_VERSION,
install_id: env.SOURCEBOT_INSTALL_ID,
$host: host,
},
distinctId,
});
const distinctId = await tryGetPostHogDistinctId();
const posthog = await createPostHogClient();

const headersList = await headers();
const host = headersList.get("host") ?? undefined;

posthog.capture({
event,
properties: {
...properties,
sourcebot_version: SOURCEBOT_VERSION,
install_id: env.SOURCEBOT_INSTALL_ID,
$host: host,
},
distinctId,
});
} catch (error) {
logger.error('Failed to capture PostHog event:', error);
}
}
Loading
Loading

Back | FazBrowse Home | New Git URL