Gentoo Portage Overlays - dev-python/tiktoken

dev-python/tiktoken

fast BPE tokeniser for use with OpenAI's models

Screenshots

https://github.com/openai/tiktoken
https://pypi.org/project/tiktoken/

tiktoken-0.7.0

~amd64

debug python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 debug

View

Download

Browse License: Apache-2.0 Apache-2.0-with-LLVM-exceptions MIT Unicode-DFS-2016 Unlicense

Overlay: powerman

ChangeLog USE Flags Dependencies Reverse Deps Related Bugs

ChangeLog

commit 7232def6bb1aeaa2e38da2d19f378b3499c16956
Author: Alex Efros <powerman@powerman.name>
Date: Mon Aug 19 19:37:22 2024 +0300

dev-python/tiktoken: new package, add 0.7.0