MidnightBSD Magustextproc/py-tokenizers
Fast state-of-the-art tokenizers optimized for research and production
| Flavor | Version | Run | OSVersion | Arch | License | Restricted | Build | Fetch | Test | Scan | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| py312 | 0.23.1_2 | 644 | 4.1 | amd64 | Apache-2.0 | 0 | untested | untested | untested | untested |
License Permissions: dist-mirror dist-sell pkg-mirror pkg-sell auto-accept
Links
Depends On
- devel/pkgconf (build)
- devel/py-build (build)
- devel/py-installer (build)
- devel/py-maturin (build)
- lang/python312 (build)
- lang/rust (build)
- devel/oniguruma (lib)
- ai/py-huggingface-hub (run)
- lang/python312 (run)
- ai/py-datasets (test)
- devel/py-pytest (test)
- lang/python312 (test)
- math/py-numpy (test)
- www/py-requests (test)
Depend Of
Categories
CVEs
- Loading CVE information...