1.0.0

Change log

0.12.0: GPT-5.1

Added

• Added support for GPT-5.1

0.11.0

Added

Added support for the following models: 04-mini, gpt-oss, gpt-4.5 and gpt-5

0.10.0

Added

• Added support for GPT-4.1 models (close #28)

Added FFI bindings for rust lib

Added an alternative implementation of the encoder using tiktoken-rs library (#27).

Added

• Added Yethee\Tiktoken\Exception\IOError class
• Added Yethee\Tiktoken\Exception\LibError class
• Added Yethee\Tiktoken\Encoder\LibEncoder class
• Added Yethee\Tiktoken\Encoder\NativeEncoder class
• Added Yethee\Tiktoken\Vocab\VocabLoader::loadFile() method

Changed

• Yethee\Tiktoken\Encoder is now an interface
• Added argument $useLib to Yethee\Tiktoken\EncoderProvider constructor

  - public function __construct()
  + public function __construct(private bool $useLib = false)

• [BC break] You can no longer disable cache for vocabs.
  Changes signature of EncoderProvider::setVocabCache() method:

  - public function setVocabCache(string|null $cacheDir): void
  + public function setVocabCache(string $cacheDir): void

0.8.0

Added

• Added support for o3 models

0.7.0

Added

• Add support o1 models (#18, #19)
• Add support chatgpt-4o models

0.6.0: Add support embedding models

Added

• Added support for models:
  • babbage-002
  • davinci-002
  • text-embedding-3-small
  • text-embedding-3-large

0.5.1

Fixed

• Fix invalidate cache of vocab when the hash is mismatch.

0.5.0

Added

• Added support for encoding o200k_base (GPT-4o) (#14)
• Invalidate cache of vocab by checksum

Changed

• [BC break] Changed signature of method VocabLoader::load

  - public function load(string $uri): Vocab;
  + public function load(string $uri, string|null $checksum = null): Vocab;