# Transformer generate function got low GPU utilization

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftransformer-generate-function-got-low-gpu-utilization%2F16600%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 7, 2022, 8:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftransformer-generate-function-got-low-gpu-utilization%2F16600 "2022-04-07T20:55:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Onlydrinkwater](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/onlydrinkwater/32/3503_2.png) [@Onlydrinkwater](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FOnlydrinkwater)%5C
**Post date:** [April 7, 2022, 8:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftransformer-generate-function-got-low-gpu-utilization%2F16600%2F1 "2022-04-07T20:55:18Z")

</div>

Hi all,  
I am now doing a bi-level training using T5 transformer. And I need to use model.generate to generate some synthetic data to train the second model for each step.  
However, when I use generate function during training, it made my GPU utilization go down to 30% from 90%. Even if I set the beam width to 1, it is still 35%.  
Doesn’t model.generate function use GPU? Why the utilization is so low.

---

<div class="post-metadata">

**Author:** ![fzyzcjy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fzyzcjy/32/25229_2.png) [@fzyzcjy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ffzyzcjy)%5C
**Post date:** [December 18, 2022, 1:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftransformer-generate-function-got-low-gpu-utilization%2F16600%2F2 "2022-12-18T01:46:35Z")

</div>

Hi, I am having the same problem. Did you find the answer? Thanks