7 Commits (c7d0dc915aae3d38d6adeeb22cfbff5cadfaf770)

Author SHA1 Message Date
  Martin Evans c7d0dc915a Assorted small changes to clean up some code warnings 1 year ago
  Martin Evans 98635a0d5a Fixed decoding of large tokens (over 16 bytes) in streaming text decoder 1 year ago
  Martin Evans 2eb52b1630 made casts to/from int explicit, fixed places affected 1 year ago
  Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 1 year ago
  Yaohui Liu cb5fb210b1
feat: optimize apis for cuda feature detection. 2 years ago
  Martin Evans a03fe003de Fixed decoding of text "accumulating" over time (never properly clearing buffer) 2 years ago
  Martin Evans 51d4411a58 Added two new classes for detokenization tasks: 2 years ago