Bug#1034307: RFP: tiktoken -- fast BPE tokeniser for use with OpenAI's models

[Petter Reinholdtsen <pere@hungry.com>]

Package: wnpp
Severity: wishlist

  Package name    : tiktoken
  Version         : 0.3.3
  Upstream Author : Shantanu Jain <shantanu@openai.com>
  URL             : https://github.com/openai/tiktoken
  License         : MIT
  Programming Lang: Python, Rust
  Description     : fast BPE tokeniser for use with OpenAI's models

I have created draft packaging rules in
<URL: https://salsa.debian.org/deeplearning-team/tiktoken.git >.

-- 
Happy hacking
Petter Reinholdtsen