scisharp
/
LLamaSharp

 
			
<!doctype html>
<html lang="en" class="no-js">
  <head>
    
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width,initial-scale=1">
      
      
        <link rel="prev" href="../save-load-state/">
      
      
        <link rel="next" href="../../ChatSession/basic-usages/">
      
      <link rel="icon" href="../../assets/images/favicon.png">
      <meta name="generator" content="mkdocs-1.4.3, mkdocs-material-9.1.16">
    
    
        <title>Differences of Executors - LLamaSharp Documentation</title>
      
    
      <link rel="stylesheet" href="../../assets/stylesheets/main.26e3688c.min.css">
      
      
        <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
        <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
        <style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
      
    
    <script>__md_scope=new URL("../..",location),__md_hash=e=>[...e].reduce((e,_)=>(e<<5)-e+_.charCodeAt(0),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
    
      
  </head>
  
  
    <body dir="ltr">
  
    
      <script>var palette=__md_get("__palette");if(palette&&"object"==typeof palette.color)for(var key of Object.keys(palette.color))document.body.setAttribute("data-md-color-"+key,palette.color[key])</script>
    
    <input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
    <input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
    <label class="md-overlay" for="__drawer"></label>
    <div data-md-component="skip">
      
        
        <a href="#differences-between-the-executors" class="md-skip">
          Skip to content
        </a>
      
    </div>
    <div data-md-component="announce">
      
    </div>
    
      <div data-md-color-scheme="default" data-md-component="outdated" hidden>
        
      </div>
    
    
<header class="md-header md-header--shadow" data-md-component="header">
  <nav class="md-header__inner md-grid" aria-label="Header">
    <a href="../.." title="LLamaSharp Documentation" class="md-header__button md-logo" aria-label="LLamaSharp Documentation" data-md-component="logo">
      
  
  <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>

    </a>
    <label class="md-header__button md-icon" for="__drawer">
      <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
    </label>
    <div class="md-header__title" data-md-component="header-title">
      <div class="md-header__ellipsis">
        <div class="md-header__topic">
          <span class="md-ellipsis">
            LLamaSharp Documentation
          </span>
        </div>
        <div class="md-header__topic" data-md-component="header-topic">
          <span class="md-ellipsis">
            
              Differences of Executors
            
          </span>
        </div>
      </div>
    </div>
    
    
      <label class="md-header__button md-icon" for="__search">
        <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
      </label>
      <div class="md-search" data-md-component="search" role="dialog">
  <label class="md-search__overlay" for="__search"></label>
  <div class="md-search__inner" role="search">
    <form class="md-search__form" name="search">
      <input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
      <label class="md-search__icon md-icon" for="__search">
        <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
        <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
      </label>
      <nav class="md-search__options" aria-label="Search">
        
        <button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
          <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
        </button>
      </nav>
      
    </form>
    <div class="md-search__output">
      <div class="md-search__scrollwrap" data-md-scrollfix>
        <div class="md-search-result" data-md-component="search-result">
          <div class="md-search-result__meta">
            Initializing search
          </div>
          <ol class="md-search-result__list" role="presentation"></ol>
        </div>
      </div>
    </div>
  </div>
</div>
    
    
  </nav>
  
</header>
    
    <div class="md-container" data-md-component="container">
      
      
      <main class="md-main" data-md-component="main">
        <div class="md-main__inner md-grid">
          
            
              <div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
                <div class="md-sidebar__scrollwrap">
                  <div class="md-sidebar__inner">
                    

<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
  <label class="md-nav__title" for="__drawer">
    <a href="../.." title="LLamaSharp Documentation" class="md-nav__button md-logo" aria-label="LLamaSharp Documentation" data-md-component="logo">
      
  
  <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>

    </a>
    LLamaSharp Documentation
  </label>
  
  <ul class="md-nav__list" data-md-scrollfix>
    
      
    <li class="md-nav__item">
      <a href="../.." class="md-nav__link">
        Overview
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../GetStarted/" class="md-nav__link">
        Get Started
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Architecher/" class="md-nav__link">
        Architecher
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Tricks/" class="md-nav__link">
        Tricks for FAQ
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../ContributingGuide/" class="md-nav__link">
        Contributing Guide
      </a>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_6" >
      
      
        <label class="md-nav__link" for="__nav_6" id="__nav_6_label" tabindex="0">
          LLamaModel
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_6_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_6">
          <span class="md-nav__icon md-icon"></span>
          LLamaModel
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../LLamaModel/parameters/" class="md-nav__link">
        Model Parameters
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../LLamaModel/tokenization/" class="md-nav__link">
        Tokenization
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../LLamaModel/embeddings/" class="md-nav__link">
        Get Embeddings
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../LLamaModel/quantization/" class="md-nav__link">
        Quantization
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../LLamaModel/save-load-state/" class="md-nav__link">
        Save/Load State
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--active md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_7" checked>
      
      
        <label class="md-nav__link" for="__nav_7" id="__nav_7_label" tabindex="0">
          LLamaExecutors
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_7_label" aria-expanded="true">
        <label class="md-nav__title" for="__nav_7">
          <span class="md-nav__icon md-icon"></span>
          LLamaExecutors
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../parameters/" class="md-nav__link">
        Inference Parameters
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../text-to-text-apis/" class="md-nav__link">
        Text-to-Text APIs
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../save-load-state/" class="md-nav__link">
        Save/Load State
      </a>
    </li>
  

    <li class="md-nav__item md-nav__item--active">
      
      <input class="md-nav__toggle md-toggle" type="checkbox" id="__toc">
      
      
        <label class="md-nav__link md-nav__link--active" for="__toc">
          Differences of Executors
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <a href="./" class="md-nav__link md-nav__link--active">
        Differences of Executors
      </a>
      
        
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
  
  
    <label class="md-nav__title" for="__toc">
      <span class="md-nav__icon md-icon"></span>
      Table of contents
    </label>
    <ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
      
        <li class="md-nav__item">
  <a href="#differences-between-the-executors" class="md-nav__link">
    Differences between the executors
  </a>
  
</li>
      
        <li class="md-nav__item">
  <a href="#interactive-mode-instruct-mode" class="md-nav__link">
    Interactive mode &amp; Instruct mode
  </a>
  
</li>
      
        <li class="md-nav__item">
  <a href="#stateful-mode-and-stateless-mode" class="md-nav__link">
    Stateful mode and Stateless mode.
  </a>
  
</li>
      
    </ul>
  
</nav>
      
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_8" >
      
      
        <label class="md-nav__link" for="__nav_8" id="__nav_8_label" tabindex="0">
          ChatSession
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_8_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_8">
          <span class="md-nav__icon md-icon"></span>
          ChatSession
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../ChatSession/basic-usages/" class="md-nav__link">
        Basic Usages
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../ChatSession/transforms/" class="md-nav__link">
        Transoforms
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../ChatSession/save-load-session/" class="md-nav__link">
        Save/Load Session
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_9" >
      
      
        <label class="md-nav__link" for="__nav_9" id="__nav_9_label" tabindex="0">
          Non-English Usages
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_9_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_9">
          <span class="md-nav__icon md-icon"></span>
          Non-English Usages
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../NonEnglishUsage/Chinese/" class="md-nav__link">
        Chinese
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_10" >
      
      
        <label class="md-nav__link" for="__nav_10" id="__nav_10_label" tabindex="0">
          High-level Applications
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_10_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_10">
          <span class="md-nav__icon md-icon"></span>
          High-level Applications
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../HighLevelApps/bot-sharp/" class="md-nav__link">
        BotSharp
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_11" >
      
      
        <label class="md-nav__link" for="__nav_11" id="__nav_11_label" tabindex="0">
          More
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_11_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_11">
          <span class="md-nav__icon md-icon"></span>
          More
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../More/log/" class="md-nav__link">
        Logger
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_12" >
      
      
        <label class="md-nav__link" for="__nav_12" id="__nav_12_label" tabindex="0">
          Examples
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_12_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_12">
          <span class="md-nav__icon md-icon"></span>
          Examples
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../Examples/ChatSessionStripRoleName/" class="md-nav__link">
        Chat session 1
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/ChatSessionWithRoleName/" class="md-nav__link">
        Chat session 2
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/GetEmbeddings/" class="md-nav__link">
        Get embeddings
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/InstructModeExecute/" class="md-nav__link">
        Instruct executor
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/InteractiveModeExecute/" class="md-nav__link">
        Interactive executor
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/StatelessModeExecute/" class="md-nav__link">
        Stateless exeutor
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/LoadAndSaveSession/" class="md-nav__link">
        Load/Save session
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/LoadAndSaveState/" class="md-nav__link">
        Load/Save state
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../Examples/QuantizeModel/" class="md-nav__link">
        Quantize model
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

    <li class="md-nav__item md-nav__item--nested">
      
      
      <input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_13" >
      
      
        <label class="md-nav__link" for="__nav_13" id="__nav_13_label" tabindex="0">
          API Reference
          <span class="md-nav__icon md-icon"></span>
        </label>
      
      <nav class="md-nav" data-md-level="1" aria-labelledby="__nav_13_label" aria-expanded="false">
        <label class="md-nav__title" for="__nav_13">
          <span class="md-nav__icon md-icon"></span>
          API Reference
        </label>
        <ul class="md-nav__list" data-md-scrollfix>
          
            
    <li class="md-nav__item">
      <a href="../../xmldocs/" class="md-nav__link">
        index
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.abstractions.ihistorytransform/" class="md-nav__link">
        llama.abstractions.ihistorytransform
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.abstractions.illamaexecutor/" class="md-nav__link">
        llama.abstractions.illamaexecutor
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.abstractions.itextstreamtransform/" class="md-nav__link">
        llama.abstractions.itextstreamtransform
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.abstractions.itexttransform/" class="md-nav__link">
        llama.abstractions.itexttransform
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.chatsession/" class="md-nav__link">
        llama.chatsession
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.authorrole/" class="md-nav__link">
        llama.common.authorrole
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.chathistory/" class="md-nav__link">
        llama.common.chathistory
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.fixedsizequeue-1/" class="md-nav__link">
        llama.common.fixedsizequeue-1
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.illamalogger/" class="md-nav__link">
        llama.common.illamalogger
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.inferenceparams/" class="md-nav__link">
        llama.common.inferenceparams
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.llamadefaultlogger/" class="md-nav__link">
        llama.common.llamadefaultlogger
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.mirostatetype/" class="md-nav__link">
        llama.common.mirostatetype
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.common.modelparams/" class="md-nav__link">
        llama.common.modelparams
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.exceptions.runtimeerror/" class="md-nav__link">
        llama.exceptions.runtimeerror
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.extensions.dictionaryextension/" class="md-nav__link">
        llama.extensions.dictionaryextension
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.instructexecutor/" class="md-nav__link">
        llama.instructexecutor
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.interactiveexecutor/" class="md-nav__link">
        llama.interactiveexecutor
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.llamaembedder/" class="md-nav__link">
        llama.llamaembedder
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.llamamodel/" class="md-nav__link">
        llama.llamamodel
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.llamaquantizer/" class="md-nav__link">
        llama.llamaquantizer
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.llamatransforms/" class="md-nav__link">
        llama.llamatransforms
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.llamacontextparams/" class="md-nav__link">
        llama.native.llamacontextparams
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.llamaftype/" class="md-nav__link">
        llama.native.llamaftype
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.llamatokendata/" class="md-nav__link">
        llama.native.llamatokendata
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.llamatokendataarray/" class="md-nav__link">
        llama.native.llamatokendataarray
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.llamatokendataarraynative/" class="md-nav__link">
        llama.native.llamatokendataarraynative
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.nativeapi/" class="md-nav__link">
        llama.native.nativeapi
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.safellamacontexthandle/" class="md-nav__link">
        llama.native.safellamacontexthandle
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.native.safellamahandlebase/" class="md-nav__link">
        llama.native.safellamahandlebase
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatcompletion/" class="md-nav__link">
        llama.oldversion.chatcompletion
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatcompletionchoice/" class="md-nav__link">
        llama.oldversion.chatcompletionchoice
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatcompletionchunk/" class="md-nav__link">
        llama.oldversion.chatcompletionchunk
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatcompletionchunkchoice/" class="md-nav__link">
        llama.oldversion.chatcompletionchunkchoice
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatcompletionchunkdelta/" class="md-nav__link">
        llama.oldversion.chatcompletionchunkdelta
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatcompletionmessage/" class="md-nav__link">
        llama.oldversion.chatcompletionmessage
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatmessagerecord/" class="md-nav__link">
        llama.oldversion.chatmessagerecord
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatrole/" class="md-nav__link">
        llama.oldversion.chatrole
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.chatsession-1/" class="md-nav__link">
        llama.oldversion.chatsession-1
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.completion/" class="md-nav__link">
        llama.oldversion.completion
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.completionchoice/" class="md-nav__link">
        llama.oldversion.completionchoice
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.completionchunk/" class="md-nav__link">
        llama.oldversion.completionchunk
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.completionlogprobs/" class="md-nav__link">
        llama.oldversion.completionlogprobs
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.completionusage/" class="md-nav__link">
        llama.oldversion.completionusage
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.embedding/" class="md-nav__link">
        llama.oldversion.embedding
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.embeddingdata/" class="md-nav__link">
        llama.oldversion.embeddingdata
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.embeddingusage/" class="md-nav__link">
        llama.oldversion.embeddingusage
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.ichatmodel/" class="md-nav__link">
        llama.oldversion.ichatmodel
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.llamaembedder/" class="md-nav__link">
        llama.oldversion.llamaembedder
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.llamamodel/" class="md-nav__link">
        llama.oldversion.llamamodel
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.oldversion.llamaparams/" class="md-nav__link">
        llama.oldversion.llamaparams
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.resettablellamamodel/" class="md-nav__link">
        llama.resettablellamamodel
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.statefulexecutorbase/" class="md-nav__link">
        llama.statefulexecutorbase
      </a>
    </li>
  

    <li class="md-nav__item">
      <a href="../../xmldocs/llama.statelessexecutor/" class="md-nav__link">
        llama.statelessexecutor
      </a>
    </li>
  

        </ul>
      </nav>
    </li>
  

  </ul>
</nav>
                  </div>
                </div>
              </div>
            
            
              <div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
                <div class="md-sidebar__scrollwrap">
                  <div class="md-sidebar__inner">
                    

<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
  
  
    <label class="md-nav__title" for="__toc">
      <span class="md-nav__icon md-icon"></span>
      Table of contents
    </label>
    <ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
      
        <li class="md-nav__item">
  <a href="#differences-between-the-executors" class="md-nav__link">
    Differences between the executors
  </a>
  
</li>
      
        <li class="md-nav__item">
  <a href="#interactive-mode-instruct-mode" class="md-nav__link">
    Interactive mode &amp; Instruct mode
  </a>
  
</li>
      
        <li class="md-nav__item">
  <a href="#stateful-mode-and-stateless-mode" class="md-nav__link">
    Stateful mode and Stateless mode.
  </a>
  
</li>
      
    </ul>
  
</nav>
                  </div>
                </div>
              </div>
            
          
            <div class="md-content" data-md-component="content">
              <article class="md-content__inner md-typeset">
                
                  
  <h1>Differences of Executors</h1>

<h2 id="differences-between-the-executors">Differences between the executors</h2>
<p>There're currently three kinds of executors provided, which are <code>InteractiveExecutor</code>, <code>InstructExecutor</code> and <code>StatelessExecutor</code>.</p>
<p>In a word, <code>InteractiveExecutor</code> is suitable for getting answer of your questions from LLM continuously. <code>InstructExecutor</code> let LLM execute your instructions, such as "continue writing". <code>StatelessExecutor</code> is best for one-time job because the previous inference has no impact on the current inference.</p>
<h2 id="interactive-mode-instruct-mode">Interactive mode &amp; Instruct mode</h2>
<p>Both of them are taking "completing the prompt" as the goal to generate the response. For example, if you input <code>Long long ago, there was a fox who wanted to make friend with humen. One day</code>, then the LLM will continue to write the story.</p>
<p>Under interactive mode, you serve a role of user and the LLM serves the role of assistant. Then it will help you with your question or request. </p>
<p>Under instruct mode, you give LLM some instructions and it follows.</p>
<p>Though the behaviors of them sounds similar, it could introduce many differences depending on your prompt. For example, "chat-with-bob" has good performance under interactive mode and <code>alpaca</code> does well with instruct mode.</p>
<pre><code>// chat-with-bob

Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.

User: Hello, Bob.
Bob: Hello. How may I help you today?
User: Please tell me the largest city in Europe.
Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.
User:
</code></pre>
<pre><code>// alpaca

Below is an instruction that describes a task. Write a response that appropriately completes the request.
</code></pre>
<p>Therefore, please modify the prompt correspondingly when switching from one mode to the other.</p>
<h2 id="stateful-mode-and-stateless-mode">Stateful mode and Stateless mode.</h2>
<p>Despite the differences between interactive mode and instruct mode, both of them are stateful mode. That is, your previous question/instruction will impact on the current response from LLM. On the contrary, the steteless executor does not have such a "memory". No matter how many times you talk to it, it will only concentrate on what you say in this time.</p>
<p>Since the stateless executor has no memory of conversations before, you need to input your question with the whole prompt into it to get the better answer.</p>
<p>For example, if you feed <code>Q: Who is Trump? A:</code> to the steteless executor, it may give the following answer with the antiprompt <code>Q:</code>.</p>
<pre><code>Donald J. Trump, born June 14, 1946, is an American businessman, television personality, politician and the 45th President of the United States (2017-2021). # Anexo:Torneo de Hamburgo 2022 (individual masculino)

## Presentación previa

* Defensor del título:  Daniil Medvédev
</code></pre>
<p>It seems that things went well at first. However, after answering the question itself, LLM began to talk about some other things until the answer reached the token count limit. The reason of this strange behavior is the anti-prompt cannot be match. With the input, LLM cannot decide whether to append a string "A: " at the end of the response.</p>
<p>As an improvement, let's take the following text as the input:</p>
<pre><code>Q: What is the capital of the USA? A: Washingtong. Q: What is the sum of 1 and 2? A: 3. Q: Who is Trump? A: 
</code></pre>
<p>Then, I got the following answer with the anti-prompt <code>Q:</code>.</p>
<pre><code>45th president of the United States.
</code></pre>
<p>At this time, by repeating the same mode of <code>Q: xxx? A: xxx.</code>, LLM outputs the anti-prompt we want to help to decide where to dtop the generation.</p>


              </article>
            </div>
          
          
        </div>
        
      </main>
      
        <footer class="md-footer">
  
  <div class="md-footer-meta md-typeset">
    <div class="md-footer-meta__inner md-grid">
      <div class="md-copyright">
  
  
    Made with
    <a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
      Material for MkDocs
    </a>
  
</div>
      
    </div>
  </div>
</footer>
      
    </div>
    <div class="md-dialog" data-md-component="dialog">
      <div class="md-dialog__inner md-typeset"></div>
    </div>
    
    <script id="__config" type="application/json">{"base": "../..", "features": [], "search": "../../assets/javascripts/workers/search.208ed371.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": {"provider": "mike"}}</script>
    
    
      <script src="../../assets/javascripts/bundle.a51614de.min.js"></script>
      
    
  </body>
</html>