Discussions

• 5 You must be logged in to vote

  🙏

  Chat history/memory

  gurbindersingh asked Feb 15, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  💬

  Does llama.cpp support CUDA through HIP?

  MarioIshac started Dec 19, 2024 in General

  

  

  0
• 3 You must be logged in to vote

  🙏

  Accelerating llama.cpp for the Copilot+PCs / Snapdragon X (updated)

  AndreasKunar asked Jul 6, 2024 in Q&A · Unanswered

  

  

  

  

  21
• 1 You must be logged in to vote

  🙏

  What does the prompt context mean?

  siddhsql asked Jun 13, 2023 in Q&A · Answered

  

  

  

  

  15
• 14 You must be logged in to vote

  🙌

  Performance of llama.cpp on Apple Silicon A-series

  ggerganov started Dec 17, 2023 in Show and tell

  

  

  

  

  38
• 1 You must be logged in to vote

  💬

  Performance of llama.cpp with Vulkan

  performance Speed related topics need feedback Testing and feedback with results are needed

  netrunnereve started Dec 18, 2024 in General

  

  

  3
• 1 You must be logged in to vote

  🙏

  is it possible to convert gpt-j to gguf?

  silvacarl2 asked Dec 17, 2024 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Embedding vectors differ between llama-server and llama-embedding

  louisjang asked Dec 18, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Requesting help with usage of llama_batch and llama_decode via Python ctypes

  ddh0 asked Dec 18, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  The question in convert llava-v1.6-mistral-7b-hf to gguf by using llama.cpp

  lyp-liu asked Dec 12, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Functional unit tests for llamafile_sgemm API.

  amritahs asked Dec 18, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Poor performance - How do I use LlamCpp in python correctly?

  rsoika asked Mar 26, 2024 in Q&A · Unanswered

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  Is it me or llama.cpp's CLI can no longer cut the chatml reverse prompt off the responses?

  l29ah asked Dec 7, 2024 in Q&A · Unanswered

  

  

  1
• 39 You must be logged in to vote

  🙌

  Performance of llama.cpp on Apple Silicon M-series

  performance Speed related topics need feedback Testing and feedback with results are needed

  ggerganov started Nov 22, 2023 in Show and tell

  

  

  

  

  159
• 6 You must be logged in to vote

  💬

  [SYCL][Intel GPU] Long Term Features & Issues Tracking

  Intel GPU

  airMeng started Feb 2, 2024 in General

  

  

  

  

  107
• 1 You must be logged in to vote

  🙏

  Why is Q4_0_4_4 support removed?

  kjh-icnl asked Dec 16, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙌

  Announcing the Edge LLM Leaderboard – Now Live with Support from Hugging Face!

  Arnav0400 started Dec 17, 2024 in Show and tell

  

  

  0
• 0 You must be logged in to vote

  🙏

  Understanding llama-bench

  aneeshmb02 asked May 10, 2024 in Q&A · Answered

  

  

  

  

  6
• 3 You must be logged in to vote

  💡

  Add support for Tenstorrent Grayskull and Wormhole PCIe cards

  cyrozap started Dec 16, 2024 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙏

  compute imatrix takes a lot of time

  DavidZyy asked Dec 14, 2024 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  Trying to run llama-server, but llama_model_load: error loading model: missing tensor 'token_embd.weight'

  MichelleTanPY asked Dec 13, 2024 in Q&A · Answered

  

  

  4
• 17 You must be logged in to vote

  💬

  About imatrix overfitting, and importance of input text

  Artefact2 started Feb 1, 2024 in General · Closed

  

  

  

  

  67
• 2 You must be logged in to vote

  🙏

  Streaming output using OpenAI Python library with llama.cpp HTTP Server in Jupyter Notebook

  qymab asked Mar 23, 2024 in Q&A · Unanswered

  

  

  

  

  7
• 1 You must be logged in to vote

  🙏

  How to Analyze Operator Latency Breakdown in llama.cpp?

  Zijie-Tian asked Dec 15, 2024 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  How to prefix the data generation

  lfoppiano started Dec 10, 2024 in General

  

  

  5