gpo.zugaina.org

Search Portage & Overlays:

dev-python/tiktoken

fast BPE tokeniser for use with OpenAI's models

Screenshots

  • tiktoken-0.7.0
    ~amd64
    debug python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 debug

    View      Download      Browse     License: Apache-2.0 Apache-2.0-with-LLVM-exceptions MIT Unicode-DFS-2016 Unlicense   
    Overlay: powerman

USE Flags

debug
Global: Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see http://www.gentoo.org/proj/en/qa/backtraces.xml
python_single_target_python3_10
* This flag is undocumented *
python_single_target_python3_11
* This flag is undocumented *
python_single_target_python3_12
* This flag is undocumented *