better_instruct_antiprompt_checking

Metal depends on this file to execute, and MacOS llama.cpp defaults is now METAL.

See on Mac OS enable Metal by default #2901

`GetState()` fix

* Added the low level bindings to beam search.

Changed `OpenOrCreate` to `Create`

- Minimal amount of characters converted
   - Allocation free
 - Added `TokensToSpan` to `SafeLlamaModelHandle` which converts as many tokens as possible into a character span
   - Allocation free

refactor: remove old version files.

The D is for my middle name 😄

Add GGUF support

Enable Semantic kernel support

- Made all the mechanics of grammar parsing (GBNFGrammarParser, ParseState) internal. Just call `Grammar.Parse("whatever")`.
 - Added a `GrammarRule` class which validates elements on construction (this allows constructing grammar without parsing GBNF).
   - It should be impossible for a `GrammarRule` to represent an invalid rule.

This commit includes changes to compile en VS Mac + changest to use llama2 not codellama.

It includes MacOS binaries in memory and metal

SymbolIds is now SortedDictionary (although I'm not sure it really needs to be) because the test was failing due to expected value being in another order. The C++ data structure if SymbolIds is std::map<std::string, uint32_t> so the items are ordered by key.