Skip to content

Pull requests: huggingface/text-generation-inference

New pull request New
20 Open 1,376 Closed
20 Open 1,376 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add fp8 kv cache for ROCm
#2856 opened Dec 18, 2024 by mht-sharma • Draft
5 tasks
Add Flash decoding kernel ROCm
#2855 opened Dec 18, 2024 by mht-sharma Loading…
5 tasks
change xpu lib download link
#2852 opened Dec 17, 2024 by sywangyi Loading…
5 tasks
2
Update Dockerfile to use devel image for compatibility
#2848 opened Dec 16, 2024 by YaserJaradeh Loading…
2 of 5 tasks
2
[TRTLLM] Expose finish reason
#2841 opened Dec 13, 2024 by mfuntowicz Loading…
1
Add possible variants for A100 and H100 GPUs for auto-detecting flops
#2837 opened Dec 13, 2024 by lazariv Loading…
5 tasks
1
bitsandbytes: upgrade and enable CUDA Graphs for 4bit by default
#2834 opened Dec 12, 2024 by matthewdouglas Loading…
4 of 5 tasks
1
2
Enable FP8 Per-Tensor Scales and Integrate Marlin/MoE Kernels Repo for ROCm
#2825 opened Dec 11, 2024 by mht-sharma Loading…
5 tasks
5
Exposes TensorRT-LLM finish reason to the server
#2818 opened Dec 10, 2024 by mfuntowicz • Draft
Flash decoding kernel adding and prefill-chunking and prefix caching enabling in intel cpu/xpu
#2815 opened Dec 10, 2024 by sywangyi Loading…
5 tasks
6
feat: tokenize each request individually and increase warmup image size
#2802 opened Dec 5, 2024 by drbh Loading…
1
Update tensor_parallel.py
#2798 opened Dec 3, 2024 by Lacacy Loading…
1
1
Install text-generation-server from poetry.lock export
#2786 opened Nov 29, 2024 by alvarobartt Loading…
1 of 5 tasks
@alvarobartt
2
Enable qwen2vl video
#2756 opened Nov 18, 2024 by drbh • Draft
6 of 9 tasks
2
Add llama.cpp backend
#2723 opened Nov 4, 2024 by mfuntowicz Loading…
13
Get opentelemetry trace id from request headers instead of creating a new trace
#2648 opened Oct 15, 2024 by kozistr Loading…
3 of 5 tasks
1
3
[WIP] Add gfx1100 support to AMD pytorch build
#2642 opened Oct 13, 2024 by cazlo • Draft
1 of 5 tasks
1
2
[DOCS] Add Google Cloud TGI integration via dedicated DLCs
#2612 opened Oct 5, 2024 by alvarobartt • Draft
1 of 5 tasks
8
Improve vlm support (add idefics3 support)
#2437 opened Aug 20, 2024 by drbh Loading…
4 tasks done
1
Add model_load_time metric
#2311 opened Jul 26, 2024 by Edwinhr716 Loading…
2 of 5 tasks
2
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.