Changelog

All notable changes to skeval are documented here.

The format follows Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

---

0.2.3 (2026-06-02)

Release highlights

• compute_metrics now raises a clear ValueError when fewer than two distinct labels are present — a 1×1 confusion matrix has no diagnostic value.

• load() raises a user-friendly FileNotFoundError instead of a raw Python traceback when the saved model directory does not exist.

• CSV and JSONL dataset loaders now validate their inputs and report errors clearly before any training begins.

• Doc-build dependencies pinned to exact versions for fully reproducible Read the Docs builds.

Fixed

• compute_metrics now raises ValueError when fewer than 2 distinct labels are found in the combined y_true / y_pred set — previously it produced a meaningless 1×1 confusion matrix and emitted a UserWarning. See issue #131 by @direkkakkar319-ops.

• load() now raises FileNotFoundError with a clear message ("No saved model found in '<path>'. Call save() first.") instead of propagating a raw Python FileNotFoundError with no context. See issue #115 by @direkkakkar319-ops.

• DatasetLoader now validates that the required text and label columns are present in CSV files before attempting to load, raising ValueError with the missing column names. See issue #121 by @iccccccccccccc.

• DatasetLoader now catches json.JSONDecodeError on malformed JSONL lines and re-raises with the line number and content for easier debugging. See issue #120 by @iccccccccccccc.

• Added wheel and build to [project.optional-dependencies] dev in pyproject.toml so python -m build --no-isolation works out of the box for contributors without extra manual installs. See issue #143 by @direkkakkar319-ops.

Changed

• Pinned exact doc-build dependency versions in docs/requirements-docs.txt for reproducible Read the Docs builds. See issue #139 by @direkkakkar319-ops.

• Bumped codecov/codecov-action from 5 to 6. By @dependabot.

• Bumped actions/upload-artifact from 4 to 7. By @dependabot.

• Bumped actions/download-artifact from 4 to 8. By @dependabot.

• Bumped MishaKav/pytest-coverage-comment from 1.0.26 to 1.7.2. By @dependabot.

Added

• Integrated CodeRabbit AI review — maintainers can trigger a review on any pull request by commenting @coderabbitai review. See issue #144 by @direkkakkar319-ops.

---

0.2.2 (2026-05-26)

Release highlights

• Deprecated train() fully replaced by fit() across all examples, scripts, and documentation — the API is now consistent with scikit-learn conventions throughout.

• assert in _batch_forward replaced with an explicit RuntimeError so unfitted-model errors survive python -O optimised mode.

Fixed

• Replace assert self.model is not None with an explicit RuntimeError in _batch_forward — assertions are silently stripped under python -O, causing an unhelpful AttributeError from PyTorch when calling predict() or predict_proba() on an unfitted model. See issue #113 by @direkkakkar319-ops.

• Replace deprecated train() with fit() in scripts/train_model.py. See issue #111 by @iccccccccccccc.

• Replace deprecated train() with fit() in examples/01_quickstart.py. See issue #112 by @iccccccccccccc.

• Replace deprecated train() with fit() in examples/02_save_and_load.py. See issue #135 by @direkkakkar319-ops.

• Replace deprecated train() with fit() in examples/03_evaluation.py. See issue #133 by @Storm1289.

• Replace deprecated train() with fit() in examples/04_custom_labels.py. See issue #134 by @direkkakkar319-ops.

• Update train() to fit() in docs/usage.rst. See issue #126 by @direkkakkar319-ops.

• Move epochs argument to SentenceClassifier constructor in docs/index.rst. See issue #127 by @direkkakkar319-ops.

Changed

• Updated repository dependencies. See issue #129 by @direkkakkar319-ops.

• Bumped actions/labeler from 5 to 6. By @dependabot.

• Bumped actions/upload-artifact from 4 to 7. By @dependabot.

Added

• Documentation link added to README.md. See issue #100 by @YuuGR1337.

• New CI/CD labels. See issue #119 by @direkkakkar319-ops.

• Test coverage for all example scripts. See issue #128 by @atharrva01.

New Contributors

• @YuuGR1337 made their first contribution.

• @iccccccccccccc made their first contribution.

• @Storm1289 made their first contribution.

• @atharrva01 made their first contribution.

---

0.2.1 (2026-05-21)

Fixed

• Updated CONTRIBUTING.md links to the correct repository URL. By @Raviteja2299.

• Replaced Sentinel AI label in CLI description, train_model script, evaluate_llm script, and 04_custom_labels example. By @taiman724.

• Fixed README clone URL and changelog repository URL. By @donglrd.

• Removed scikit-learn boilerplate from PR template. By @donglrd.

New Contributors

• @Raviteja2299 made their first contribution.

• @taiman724 made their first contribution.

• @donglrd made their first contribution.

---

0.2.0 (2026-05-19)

Release highlights

• New model_selection module with train_test_split() and cross_val_score().

• predict_proba() — probability outputs for LIME, SHAP, and ONNX compatibility.

• SentenceClassifier now inherits from sklearn.base.BaseEstimator, enabling full sklearn pipeline and GridSearchCV compatibility.

Added

• model_selection — new module with train_test_split() and cross_val_score(). By @direkkakkar319-ops.

• predict_proba() — probability outputs for LIME, SHAP, and ONNX compatibility. By @direkkakkar319-ops.

• Validation split and early stopping in fit() (val_split, patience parameters). By @direkkakkar319-ops.

• Batched prediction in predict() and predict_proba(). By @direkkakkar319-ops.

• num_workers and pin_memory parameters on the DataLoader for faster data loading. By @direkkakkar319-ops.

• random_state parameter on SentenceClassifier for reproducible training. By @direkkakkar319-ops.

• Input validation in fit() and predict(). By @direkkakkar319-ops.

• SentenceClassifier now inherits from sklearn.base.BaseEstimator — fully compatible with sklearn pipelines and GridSearchCV. By @direkkakkar319-ops.

• check_estimator() compliance tests in CI. By @direkkakkar319-ops.

• Integration tests for the full pipeline (CSV → train → save → load → predict → evaluate). By @direkkakkar319-ops.

• Test suite grew from 11 to 79 tests. By @direkkakkar319-ops.

• Google-style docstrings on all public classes and functions. By @direkkakkar319-ops.

• Ecosystem compatibility documentation (LIME, SHAP, ONNX, skore, GridSearchCV). By @direkkakkar319-ops.

• Read the Docs configuration and Sphinx docs build workflow. By @direkkakkar319-ops.

Changed

• VocabBuilder now builds word2idx and idx2word in a single pass. By @direkkakkar319-ops.

• transformers and datasets moved to optional extras (pip install skeval[transformers]). By @direkkakkar319-ops.

• Dependency upper bounds pinned to prevent silent breakage. By @direkkakkar319-ops.

• Full type annotations with mypy --strict enforced in CI. By @direkkakkar319-ops.

---

0.1.2 (2026-04-25)

Changed

• Renamed the library to skeval. By @direkkakkar319-ops.

Fixed

• Miscellaneous fixes and stability improvements. By @direkkakkar319-ops.

---

0.1.1 (2026-04-25)

Fixed

• CI workflow: updated actions/checkout to v4 and actions/setup-python to v5 (v6 does not exist and caused CI failures). By @direkkakkar319-ops.

• README: corrected predict() usage example — method takes a list of strings, not a single string. By @direkkakkar319-ops.

• README: corrected example output keys (per_class_f1 → per_class). By @direkkakkar319-ops.

• README: fixed install URL placeholder (your-username → correct repository path). By @direkkakkar319-ops.

---

0.1.0 (2026-04-25)

Release highlights

First public release of skeval — a lightweight semantic evaluation layer for LLMs.

Added

• SentenceClassifier — train, predict, save, and load a PyTorch sentence classifier. By @direkkakkar319-ops.

• BasicTextClassifier — EmbeddingBag + Linear neural network architecture. By @direkkakkar319-ops.

• Evaluator — evaluate predicted labels against ground truth. By @direkkakkar319-ops.

• compute_metrics() — accuracy, per-class precision / recall / F1, and confusion matrix via scikit-learn. By @direkkakkar319-ops.

• DatasetLoader — load training data from CSV or JSON Lines files. By @direkkakkar319-ops.

• SentenceDataset — PyTorch Dataset wrapper with variable-length collation. By @direkkakkar319-ops.

• VocabBuilder — bag-of-words tokenizer with <PAD> / <UNK> support. By @direkkakkar319-ops.

• LabelEncoder — string label ↔ integer index mapping. By @direkkakkar319-ops.

• scripts/train_model.py — CLI script for training from file. By @direkkakkar319-ops.

• scripts/evaluate_llm.py — CLI script for evaluation from file. By @direkkakkar319-ops.

• Sphinx documentation. By @direkkakkar319-ops.

• Full pytest test suite. By @direkkakkar319-ops.

• 5 example scripts in examples/. By @direkkakkar319-ops.