`options.pipeline_options`

source

The pipeline_options.py module contains the LLMModel enum and PipelineOptions class, which define the configuration options for the drug name conversion pipeline. This file provides a Pydantic model that can be used for API requests, with defaults matching those in BaseOptions.

`LLMModel`

class LLMModel()

This enum holds the names and details of the different models the assistant can use. Each variant is a tuple containing

name: The string used to define the variant in the environment
ollama_spec: The string used to call the model in Ollama
repo_id: The repository ID in huggingface
filename: The filename in the huggingface repository

Methods

`get_eot_token`

def get_eot_token()

Some models need a special token to be appended to a prompt. If so, this returns the right end-of-turn token.

`PipelineOptions`

class PipelineOptions(
  llm_model: LLMModel = LLMModel.LLAMA_3_1_8B
  temperature: float = 0
  vocabulary_id: list[str] = ["RxNorm"]
  concept_ancestor: bool = False
  concept_relationship: bool = False
  concept_synonym: bool = False
  search_threshold: int = 80
  max_separation_descendants: int = 1
  max_separation_ancestor: int = 1
  embeddings_path: str = "concept_embeddings.qdrant"
  force_rebuild: bool = False
  embed_vocab: list[str] = ["RxNorm", "RxNorm Extension"]
  embedding_model: EmbeddingModelName = EmbeddingModelName.BGESMALL
  embedding_search_kwargs: dict = {}
)

This class holds the options available to the lettuce pipeline.

These are all the options in the BaseOptions parser. The defaults provided here match the default options in BaseOptions. Using a pydantic model means FastAPI can take these as input in the API request.

Attributes

llm_model: LLMModel The name of the LLM used in the pipeline. The permitted values are the possibilities in the LLMModel enum.

temperature: float Temperature supplied to the LLM that tunes the variability of responses.

concept_ancestor: bool (Defaults to false) If true, the concept_ancestor table of the OMOP vocabularies is queried for the results of an OMOP search.

concept_relationship: bool (Defaults to false) If true, the concept_relationship table of the OMOP vocabularies is queried for the results of an OMOP search.

concept_synonym: bool (Defaults to false) If true, the concept_synonym table of the OMOP vocabularies is queried when OMOP concepts are fetched.

search_threshold: int The threshold on fuzzy string matching for returned results.

max_separation_descendant: int The maximum separation to search for concept descendants.

max_separation_ancestor: int The maximum separation to search for concept ancestors

Implemented models

Model name	Summary
llama-3.1-8b	Recommended Meta’s Llama 3.1 with 8 billion parameters, quantized to 4 bits
llama-2-7b-chat	Meta’s Llama 2 with 7 billion parameters, quantized to 4 bits
llama-3-8b	Meta’s Llama 3 with 8 billion parameters, quantized to 4 bits
llama-3-70b	Meta’s Llama 3 with 70 billion parameters, quantized to 4 bits
gemma-7b	Google’s Gemma with 7 billion parameters, quantized to 4 bits
llama-3.2-3b	Meta’s Llama 3.2 with 3 billion parameters, quantized to 6 bits
mistral-7b	Mistral at 7 billion parameters, quantized to 4 bits
kuchiki-l2-7b	A merge of several models at 7 billion parameters, quantized to 4 bits
tinyllama-1.1b-chat	Llama 2 extensively pre-trained, with 1.1 billion parameters, quantized to 4 bits
biomistral-7b	Mistral at 7 billion parameters, pre-trained on biomedical data, quantized to 4 bits
qwen2.5-3b-instruct	Alibaba’s Qwen 2.5 at 3 billion parameters, quantized to 5 bits
airoboros-3b	Llama 2 pre-trained on the airoboros 3.0 dataset at 3 billion parameters, quantized to 4 bits
medicine-chat	Llama 2 pre-trained on medical data, quantized to 4 bits
medicine-llm-13b	Llama pre-trained on medical data at 13 billion parameters, quantized to 3 bits
med-llama-3-8b-v1	Llama 3 at 8 billion parameters, pre-trained on medical data, quantized to 5 bits
med-llama-3-8b-v2	Llama 3 at 8 billion parameters, pre-trained on medical data, quantized to 4 bits
med-llama-3-8b-v3	Llama 3 at 8 billion parameters, pre-trained on medical data, quantized to 3 bits
med-llama-3-8b-v4	Llama 3 at 8 billion parameters, pre-trained on medical data, quantized to 3 bits

If you would like to add a model, raise an issue

Base Options Routers