adding extrinsic eval notebook - #374
Conversation
Sorry, something went wrong.
Signed-off-by: Eric Pham-Hung <ephamhung@ephamhung-mlt.client.nvidia.com>
Signed-off-by: Eric Pham-Hung <ephamhung@nvidia.com>
Signed-off-by: Eric Pham-Hung <ephamhung@nvidia.com>
nina-xu
left a comment
There was a problem hiding this comment.
overall it looks good! thanks so much for putting this together
Sorry, something went wrong.
| "id": "630e3e17", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# 🎛️ NeMo Safe Synthesizer 101: Extrinsic Evaluation\n", |
There was a problem hiding this comment.
102?
Sorry, something went wrong.
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# This script defines a scikit-learn pipeline for a classification task.\n", |
There was a problem hiding this comment.
For the extrinsic evaluation portion, there’s a bit of code repetition. Suggest to DRY it up by defining the train + eval steps into a function, and call that function twice with train_and_evaluate_logistic_regression(df, test_df); train_and_evaluate_logistic_regression(synthetic_df, test_df). This also makes it very clear to a user what we are doing here.
Sorry, something went wrong.
| "from sklearn.metrics import classification_report, accuracy_score, roc_auc_score\n", | ||
| "\n", | ||
| "original_pipeline = full_pipeline \n", | ||
| "print(\"\\n--- Training Benchmark Model on Original Data (1000 rows) ---\")\n", |
There was a problem hiding this comment.
I don't think the 1000 here is accurate here?
Sorry, something went wrong.
| "| Accuracy | 0.9404 | 0.9278 |\n", | ||
| "| ROC AUC Score | 0.9782 | 0.9762 |\n", | ||
| "| Precision (Class 1) | 0.9626 | 0.9423 |\n", | ||
| "| Recall (Class 1) | 0.9646 | 0.9714 |\n", |
There was a problem hiding this comment.
This is amazing results. out of curiorsity what was the SQS?
Sorry, something went wrong.
Reviewers
1 more reviewerAt least 1 approving review is required to merge this pull request.
No description provided.