VibeChecker

What it does

VibeChecker is a Chrome extension that reads the comments on a YouTube video and tells you the vibe: how many are positive, neutral and negative. A scikit-learn model served by FastAPI does the classifying, and MLflow tracks every experiment.

How it works

1. The extension collects the comments currently loaded on the page.
2. The background worker sends them in one batch to the API.
3. The API cleans the text and predicts negative, neutral or positive for each comment.
4. The popup shows the percentage split.

Model

Trained on about 37k labeled Reddit comments with three classes (negative, neutral, positive).

Model	Accuracy	Macro F1
Baseline (TF-IDF + Logistic Regression, C=1)	0.8474	0.8376
Tuned (Logistic Regression, C=10, word unigrams)	0.8929	0.8832

The tuned model came from a 25 trial random search over the vectorizer and classifier settings. Models were selected on a validation split and scored once on a held-out test split. Every trial is logged in MLflow.

Run it

Start the API with Docker

docker run -p 8000:8000 ghcr.io/malahimhaseeb/vibechecker:latest

Or run it from source

You need Python 3.10 or newer.

git clone https://github.com/MalahimHaseeb/vibechecker.git
cd vibechecker
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python -m uvicorn api.main:app

On Windows activate with .venv\Scripts\activate.

Load the extension

1. Open chrome://extensions
2. Turn on Developer mode
3. Click Load unpacked and select the extension folder
4. Open a YouTube video, scroll down until comments load, click the VibeChecker icon and press the button

YouTube loads comments as you scroll, so only the comments already on screen are analyzed.

API

GET /health returns a simple status check. POST /predict takes a list of comments and returns a label for each one plus the totals.

curl -X POST http://localhost:8000/predict -H "Content-Type: application/json" -d '{"comments": ["this video is amazing", "worst video ever"]}'

{
  "labels": ["positive", "negative"],
  "counts": {"negative": 1, "neutral": 0, "positive": 1},
  "total": 2
}

Interactive docs are at http://localhost:8000/docs while the API is running.

Limitations

• The model is trained on Reddit text, and YouTube comments use different slang, emojis and shorthand, so accuracy will be lower than the numbers above.
• English only.
• Only comments loaded on the page are analyzed.

Roadmap

• Retraining workflow with a metric gate before promoting a new model
• Hosted API with HTTPS and a Chrome Web Store release
• A small hand-labeled YouTube evaluation set