FastAPI · Next.js · SQLite · Groq (Llama 3) · Plotly · scikit-learn · XGBoost

An AI-powered analytics assistant that ingests messy real-world datasets, automatically analyzes data quality, generates its own Python analysis code on the fly, runs machine learning model comparisons, and produces polished PDF reports — all from natural language questions.

• Screenshots
• How It Works
• Architecture
• Quick Start (Windows)
• Project Structure
• API Endpoints
• What You Can Ask the Chat
• Data Quality & Ingestion
• Engineering Challenges Solved
• Environment Variables
• Tech Stack
• License

Unlike a keyword-routed chatbot, CortexIQ's chat is a code-generation and execution pipeline:

1. You ask a question in plain English ("show me a histogram of age", "predict charges", "cluster this data").
2. Groq (Llama 3) generates real pandas/Plotly/scikit-learn Python code, using the dataset's exact column names and dtypes as context.
3. The code runs in a sandboxed executor (restricted builtins, no imports) against your actual dataframe.
4. If the code errors, CortexIQ automatically sends the error back to the LLM for up to 2 self-correction attempts before giving up.
5. Results — a chart, a table, or a full ML model comparison — stream back to the browser over Server-Sent Events.

flowchart TD
    subgraph Client["Frontend"]
        FE[Next.js UI]
    end

    subgraph Backend["FastAPI Backend"]
        API[API Routes]
        ING["Ingestion Layer<br/>delimiter/encoding recovery,<br/>sentinel-value detection"]
        EXEC[Sandboxed Code Executor]
        MLENG["ML Engine<br/>anomaly · clustering · model comparison"]
        RPT["PDF Report Generator<br/>ReportLab + Kaleido"]
    end

    GROQ["Groq LLM<br/>(Llama 3)"]
    DB[(SQLite)]

    FE -- "1. Upload dataset" --> API
    API --> ING
    ING --> DB

    FE -- "2. Ask a question (SSE)" --> API
    API -- "3. Generate code" --> GROQ
    GROQ -- "Python code" --> EXEC
    EXEC -- "4. Error → retry with feedback" --> GROQ
    EXEC -- "5. Success" --> MLENG
    EXEC --> RPT
    MLENG -- "chart / table / downloadable model" --> API
    RPT -- "branded PDF" --> API
    API -- "Stream results" --> FE

• Python 3.10+
• Node.js 18+
• A free Groq API key

REM Run from the CortexIQ/ root folder
setup.bat

Or manually:

REM Terminal 1 — Backend
cd backend
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
copy .env.example .env
REM Edit .env and add your GROQ_API_KEY
uvicorn app.main:app --reload --port 8000

REM Terminal 2 — Frontend
cd frontend
npm install
npm run dev

Open http://localhost:3000 in your browser.

CortexIQ/
├── backend/
│   ├── app/
│   │   ├── main.py                  # FastAPI app entry point
│   │   ├── core/
│   │   │   ├── config.py            # Settings (pydantic-settings)
│   │   │   └── security.py          # JWT auth utilities
│   │   ├── db/
│   │   │   └── database.py          # SQLAlchemy engine + session
│   │   ├── models/                  # SQLAlchemy ORM models
│   │   │   ├── user.py
│   │   │   ├── project.py
│   │   │   ├── dataset.py
│   │   │   └── conversation.py
│   │   ├── schemas/                 # Pydantic request/response schemas
│   │   │   ├── auth.py
│   │   │   └── project.py
│   │   ├── api/routes/              # FastAPI routers
│   │   │   ├── auth.py              # /auth/register, /auth/login, /auth/me
│   │   │   ├── projects.py          # /projects CRUD
│   │   │   ├── datasets.py          # upload, quality, preview, PDF report
│   │   │   ├── chat.py              # /conversations + SSE code-execution chat
│   │   │   └── ml.py                # dedicated ML task endpoints
│   │   └── services/
│   │       ├── ingestion.py         # Robust parsing: encoding detection,
│   │       │                        #   delimiter/decimal recovery, sentinel-
│   │       │                        #   value (e.g. -200) detection
│   │       ├── analytics.py         # Descriptive stats, correlation, charts
│   │       ├── code_executor.py     # Sandboxed execution of LLM-generated code
│   │       ├── ml_engine.py         # Anomaly detection, clustering, and
│   │       │                        #   classifier/regressor comparison —
│   │       │                        #   saves a self-contained, downloadable
│   │       │                        #   sklearn Pipeline (preprocessing baked in)
│   │       ├── report_generator.py  # One-click branded PDF report (ReportLab
│   │       │                        #   + Plotly/Kaleido), with auto-generated
│   │       │                        #   insights and outlier/correlation charts
│   │       └── llm.py               # Groq LLM service (code-gen + chat)
│   ├── requirements.txt
│   └── .env.example
│
├── frontend/
│   └── src/
│       ├── app/
│       │   ├── login/page.tsx               # Auth page
│       │   ├── dashboard/
│       │   │   ├── layout.tsx               # Auth guard + sidebar
│       │   │   ├── page.tsx                 # Projects list
│       │   │   └── projects/[projectId]/page.tsx  # Project workspace
│       ├── components/
│       │   ├── dashboard/
│       │   │   ├── Sidebar.tsx
│       │   │   ├── DatasetUploader.tsx      # Drag-and-drop upload
│       │   │   └── DatasetCard.tsx          # Dataset status card
│       │   └── chat/
│       │       ├── ChatWindow.tsx           # Full chat UI, SSE streaming,
│       │       │                            #   report + model download buttons
│       │       └── ChatMessage.tsx          # Message bubble + Plotly chart +
│       │                                    #   downloadable model button
│       ├── lib/
│       │   ├── api.ts               # Axios API client + SSE streaming
│       │   └── store.ts             # Zustand auth store
│       └── types/index.ts
│
├── setup.bat                        # Windows one-click setup
└── README.md

Additional ML-specific endpoints live in app/api/routes/ml.py. Full interactive docs at http://localhost:8000/docs.

Since chat is powered by code generation rather than fixed keywords, phrasing is flexible. Examples that map to specific backend capabilities:

ingestion.py is built to survive messy real-world files, not just clean CSVs:

• Auto-detects delimiter (comma, semicolon, tab) and encoding
• Recovers comma-decimal numeric columns (e.g. European exports: "13,6" → 13.6)
• Strips stray trailing empty columns from artifacts like a;b;c;;
• Flags likely missing-data sentinel codes (e.g. -200, 9999) that would otherwise silently skew statistics — without guessing and auto-replacing them, since that's a domain judgment call left to the user
• Surfaces every parsing decision as a visible warning rather than failing (or worse, succeeding) silently

A few real, non-obvious bugs found and fixed during development — included here because "it works" and "it's correct" aren't always the same thing.

Problem: the ingestion layer tries multiple parsing strategies (comma/semicolon/tab, various quoting rules) before giving up — but on real-world files, every attempt was quietly producing a single garbage column instead of properly parsed data. Root cause: each strategy explicitly passed low_memory=False, which pandas silently rejects the moment engine="python" is also set — it raises a ValueError immediately, before any real parsing happens. A blanket except: continue was catching that error and hiding it, letting a much worse fallback strategy "succeed" instead. Fix: only pass low_memory when using the C engine, and surface the real per-strategy exception instead of swallowing it — turning an invisible failure into a debuggable one.

Problem: the automated PDF report generator produced reports with every chart slot empty — no error, just missing images — even though the exact same chart-generation code worked perfectly when run as a standalone script. Root cause: Kaleido's headless Chromium export calls signal.signal() internally, which Python only permits from the main thread. FastAPI/Starlette runs synchronous route handlers in a background thread pool, not the main thread — so every chart export was failing there with a permissions error that the report generator's broad exception handling was quietly absorbing. Fix: converted the report-generation route to async def, which runs directly on the event loop (main thread) instead of the thread pool — resolving the conflict without touching the chart code itself.

Problem: the "download trained model" feature produced a working .joblib file — but loading it and running it on new raw data either crashed or silently mispredicted. Root cause: categorical columns were label-encoded with a fresh LabelEncoder per column at training time, and only the raw fitted estimator was serialized afterward. The encoders themselves — and the exact mapping they'd learned — were discarded, making the saved model impossible to use correctly on new data without reverse-engineering the encoding by hand. Fix: rebuilt training as a single sklearn.Pipeline (ColumnTransformer + estimator), so preprocessing and the model are serialized together as one self-contained artifact. Verified with adversarial tests — an unseen category, a missing value — to confirm the pipeline degrades gracefully instead of failing.

Problem: some real-world datasets encode missing readings as sentinel numbers (e.g. -200) instead of true nulls — which silently distorts every downstream statistic (means, correlations) without pandas ever flagging the column as having missing data. Approach: built a generic detector that flags a column when one value is both suspiciously overrepresented (≥3% of rows) and a statistical outlier relative to the rest of that column's distribution (beyond 3×IQR) — a strong signature of a placeholder code. No specific sentinel value is hardcoded, so it generalizes across datasets instead of special-casing one file.

Edit backend/.env (see .env.example):

SECRET_KEY=your-secret-key-change-this
DATABASE_URL=sqlite:///./cortexiq.db
GROQ_API_KEY=your-groq-api-key
GROQ_MODEL=llama-3.1-70b-versatile

This project is licensed under the MIT License — see LICENSE for details.

Built by Muhammad Ali Ejaz — BS Data Science, IUB Bahawalpur 2023–2027