Open-source annotation system for education transcript data, including:

• human annotation workspaces,
• admin assignment and quality-control tooling,
• optional LLM-generated notes,
• structured post-annotation scavenger-hunt reflection,
• transcript media and instructional material management.

Built with Next.js App Router, Clerk authentication, Prisma + Postgres, and Google Cloud Storage.

• Why EduCoder
• Paper Alignment
• Core Features
• System Architecture
• Tech Stack
• Repository Structure
• Data Model Overview
• Prerequisites
• Environment Variables
• Local Development Setup
• Running The App
• Authentication And Roles
• Transcript File Requirements
• Storage And Upload Workflows
• LLM Annotation Workflow
• Deployment Notes
• Troubleshooting
• Contributing
• Roadmap Ideas
• Citation
• License

EduCoder is designed for research and practitioner teams working with classroom conversation transcripts that need a repeatable process to:

• assign annotation work,
• capture evidence-based notes tied to specific transcript lines,
• compare human and LLM-generated annotations,
• administer structured follow-up tasks (scavenger hunts),
• manage associated instructional context and media.

The platform uses workspace-level data boundaries so each team can work independently.

This repository is aligned with the framing in the EduCoder research paper:

• Integrated annotation workspace
  • Transcript text, synchronized video, and instructional context are available in a single workspace.
• Segment-aware navigation
  • Annotation can be scoped using transcript metadata (segment columns and optional timing cues), reducing manual clipping workflows.
• Team-based workflows
  • Role-aware admin/annotator flows support assignment, monitoring, and export.
• Human-LLM collaboration
  • LLM-generated reference notes can be generated with prompt controls and revealed using admin-configurable visibility rules.
• Structured reflection after annotation
  • The scavenger-hunt module supports post-annotation comparison and reflection on human vs. LLM interpretations.

Public access and demo links referenced in the paper:

• App: https://edu-coder.com

• Annotator workspace

  • Browse assigned transcripts in /workspace
  • Open detailed annotation interface in /annotate
  • Filter transcript lines, flag lines, assign notes to lines
  • View instructional cards and linked transcript video
  • Mark annotations complete

• Admin dashboard

  • Manage transcripts, annotators, annotations, LLM annotations, scavenger hunts, and videos under /admin/*
  • Upload transcript files and associated reference files
  • Upload or replace transcript-linked videos
  • Download generated artifacts (annotation files, note bundles, submissions)

• LLM note generation

  • Uses OpenAI Responses API via server route
  • Supports customizable prompt templates + static prompt components
  • Supports line-range scoping per transcript
  • Includes workspace-level LLM usage quota controls
  • Configurable LLM visibility defaults and per-annotator overrides to reduce annotation bias risk

• Scavenger hunts

  • Admins define question sets per transcript
  • Assignments tracked per annotator
  • Visibility controls for admin/user/per-annotator cases
  • Structured post-annotation reflection with exportable submissions

• Cloud storage integration

  • Transcript files, reference files, instructional materials, and videos stored in Google Cloud Storage
  • Supports direct server uploads and signed URL video uploads

• Frontend/UI: Next.js App Router pages (src/app/**/page.tsx) with React client components for interactive workflows.
• Backend/API: Route handlers under src/app/api/**/route.ts.
• Data layer: Prisma Client with Neon adapter and PostgreSQL schema in prisma/schema.prisma.
• Auth: Clerk middleware + server auth checks on API routes and protected layouts.
• Blob/file storage: Google Cloud Storage through helper utilities in src/app/api/admin/transcripts/storage.ts.
• Prompt assets: Prompt fragments in prompts/ used during LLM generation.

• Runtime: Node.js (recommended: active LTS)
• Framework: Next.js 16 + React 19 + TypeScript
• Styling: Tailwind CSS
• Auth: Clerk (@clerk/nextjs)
• ORM/DB: Prisma 7 + PostgreSQL (@prisma/adapter-neon)
• Storage: @google-cloud/storage
• LLM integration: OpenAI Responses API (HTTP request from server route)
• File parsing: papaparse, xlsx, jszip

.
├── prisma/
│   ├── schema.prisma
│   └── migrations/
├── prompts/
│   ├── note_creation_prompt_part_1_customizable.md
│   └── note_creation_prompt_part_2_static.md
├── public/
├── src/
│   ├── app/
│   │   ├── api/
│   │   ├── admin/
│   │   ├── annotate/
│   │   ├── login/
│   │   └── workspace/
│   ├── components/
│   ├── context/
│   └── lib/
├── LICENSE
└── README.md

Major Prisma entities:

• Workspace: top-level tenant boundary
• User: app user mapped to Clerk auth identity (auth_user_id)
• Transcripts: transcript metadata, storage paths, visibility settings
• TranscriptLines: parsed line-by-line transcript content
• TranscriptSegments: optional segment grouping for transcript lines
• Annotations: assignment/annotation records per transcript + annotator
• Notes + NoteAssignments: user and LLM notes, linked to transcript lines
• FlagAssignments: user-flagged lines
• InstructionalMaterial: per-transcript supporting image assets
• Videos: transcript-associated video objects
• ScavengerHunt* models: scavenger question/assignment/answer workflows
• LLMNotePrompts: per-transcript LLM prompt settings and line-range config

Before running locally, ensure you have:

• Node.js 20+ (or current LTS)
• npm 10+
• PostgreSQL database (Neon recommended but not required)
• Clerk project (for auth keys and app URL configuration)
• Google Cloud Storage bucket + service account with object read/write permissions
• OpenAI API key (if using LLM annotation features)

Create a local .env file in the project root.

Example:

# Database
DATABASE_URL="postgresql://USER:PASSWORD@HOST:5432/DB?sslmode=require"

# Clerk
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY="pk_test_..."
CLERK_SECRET_KEY="sk_test_..."

# Google Cloud Storage
GOOGLE_CLOUD_STORAGE_BUCKET="your-bucket-name"
# Optional aliases supported by code:
# GCS_BUCKET_NAME="your-bucket-name"
# GOOGLE_STORAGE_BUCKET="your-bucket-name"

# IMPORTANT: this code expects raw JSON content, not a file path.
# Escape newlines as \n when storing in plain env files.
GOOGLE_APPLICATION_CREDENTIALS='{"type":"service_account","project_id":"...","private_key":"-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n","client_email":"..."}'

# Optional if not present in GOOGLE_APPLICATION_CREDENTIALS:
GOOGLE_CLOUD_PROJECT_ID="your-gcp-project-id"

# LLM features
OPENAI_API_KEY="sk-..."

Notes:

• DATABASE_URL is required at startup.
• CLERK_SECRET_KEY is required for auth-user provisioning and annotator management routes.
• OPENAI_API_KEY is required only for LLM note generation routes.

1. Install dependencies

npm install

2. Apply Prisma migrations

npx prisma migrate deploy

For local schema changes during development:

npx prisma migrate dev

3. Start the app

npm run dev

4. Open:

• http://localhost:3000

• npm run dev - start dev server
• npm run build - generate Prisma client then build Next.js
• npm start - run production build
• npm run lint - run ESLint over TS/TSX files

EduCoder uses Clerk for identity and an internal User table for app-level roles.

Roles in schema:

• admin
• annotator
• llm (system/support role)

Flow:

• / serves login UI.
• Clerk session is validated by middleware/protected routes.
• /api/auth/ensure-user provisions first-time users in the app database and creates a workspace.
• Admin routes additionally enforce publicMetadata.role === 'admin'.

Accepted transcript upload file types:

• .csv
• .xls
• .xlsx

Required columns (header matching is flexible/case-insensitive):

• line number (Line Number, #, etc.)
• Speaker
• utterance/dialogue (Utterance or Dialogue)

Optional columns:

• segment (Segment)
• timing cues (In Cue, Out Cue) including decimal seconds or SMPTE-like values.

If headers are missing or data rows are invalid, upload routes return validation errors.

Storage helpers live in src/app/api/admin/transcripts/storage.ts.

Objects stored include:

• transcript source files
• associated/reference annotation files
• instructional material files
• transcript video files

Video upload options:

• stream upload through backend route
• signed URL flow with metadata validation and completion callback

When replacing existing video assets, previous objects are cleaned up best-effort.

Signed uploads from browsers require bucket CORS allowing PUT and required x-goog-meta-* headers.

Example policy:

[
  {
    "origin": ["https://www.edu-coder.com", "http://localhost:3000"],
    "method": ["PUT"],
    "responseHeader": ["Content-Type", "x-goog-meta-*"],
    "maxAgeSeconds": 3600
  }
]

Apply:

gsutil cors set cors.gcs.json gs://YOUR_BUCKET_NAME

You can also adapt the included cors.gcs.json file in this repo.

LLM note generation endpoint:

• POST /api/admin/transcripts/[transcriptId]/llm-notes/generate

High-level flow:

1. Validate actor and transcript ownership
2. Build transcript prompt payload (full transcript or configured line range)
3. Merge customizable + static prompt components from prompts/
4. Request structured output from OpenAI Responses API
5. Parse and normalize notes + note-to-line assignments
6. Persist notes/assignments and update transcript status/usage counters

Related capabilities:

• per-transcript prompt settings
• downloadable LLM note outputs
• admin default visibility + per-annotator visibility controls

Design note:

• The implementation supports using LLM outputs as optional reference annotations; human annotation remains primary in the workflow.

• Runtime is Node.js (export const runtime = 'nodejs' in API routes).
• Ensure all required environment variables are configured in deployment platform settings.
• If deploying behind a custom domain, update Clerk allowed URLs and redirect URLs.
• Verify GCS bucket IAM + CORS in production.
• Run migrations in CI/CD before app startup (npx prisma migrate deploy).

• DATABASE_URL is not set

  • Add DATABASE_URL to .env and restart.

• Clerk routes returning unauthorized/forbidden

  • Check Clerk keys, middleware matcher behavior, and user role metadata.

• GOOGLE_APPLICATION_CREDENTIALS parse errors

  • Value must be valid JSON string content (not file path), with escaped newlines in private key.

• Upload failures from browser

  • Confirm bucket CORS and correct signed upload headers.

• LLM generation fails

  • Check OPENAI_API_KEY, prompt files in prompts/, workspace quota, and API route logs.

Contributions are welcome.

Recommended flow:

1. Fork the repo
2. Create a feature branch
3. Make focused changes with clear commit messages
4. Run:
   • npm run lint
   • npm run build
5. Open a pull request with:
   • problem statement
   • implementation summary
   • screenshots/GIFs for UI changes
   • test/verification notes

Suggested PR checklist:

• no secrets committed
• env var changes documented
• migration changes included if schema changed
• backward compatibility impact noted

• Automated test coverage (unit + integration + route tests)
• Role-scoped audit logs in admin panel
• Better prompt versioning and evaluation workflow for LLM notes
• Bulk assignment/import workflows
• Production monitoring/observability dashboards

This project is licensed under the MIT License. See LICENSE for full text.