Apocrypha

tokenizers

Implementation of today's most used tokenizers

Available in

OverlayNewestEbuildsLast activity
gentoo gitweb ↗ 0.23.1 3 18 h details ›
anonch GitHub ↗ 0.21.1-r88881 3 2 d details ›

Versions & arches

VersionOverlay amd64 Committed
0.23.1 gentoo amd64 testing view · download · history ↗
0.22.2 gentoo amd64 testing view · download · history ↗
0.22.1 gentoo amd64 testing view · download · history ↗
0.21.1-r88881 anonch amd64 testing 14 mo view · download · history ↗
0.21.1-r1 anonch amd64 testing 14 mo view · download · history ↗
0.21.0 anonch amd64 testing 14 mo view · download · history ↗

Use flags of 0.23.1

  • test Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently)
  • debug Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
3 expansion flags (python targets, ABIs, cpu flags…)
  • python_single_target_python3_12
  • python_single_target_python3_13
  • python_single_target_python3_14

Runtime dependencies of 0.23.1

show 10 lines
python_single_target_python3_12? (
)
python_single_target_python3_13? (
)
python_single_target_python3_14? (
)