# How are the inputs tokenized when model deployment?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-are-the-inputs-tokenized-when-model-deployment%2F9692%3E%5C
**Category:** Amazon SageMaker\
**Created:** [September 2, 2021, 1:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-are-the-inputs-tokenized-when-model-deployment%2F9692 "2021-09-02T13:39:10Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fphilschmid)%5C
**Post date:** [September 2, 2021, 3:19pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-are-the-inputs-tokenized-when-model-deployment%2F9692%2F6 "2021-09-02T15:19:46Z")

</div>

Training and Inference are two completely different things. You are using the same tokenizer, but not the same configuration.

First of all, it is not possible to predict a longer sentence than `512` with the model you use. Meaning you can either use a model, which supports a longer input sequence, e.g. [longofrmer](https://huggingface.co/allenai/longformer-base-4096) or you can truncate your inputs in advance, so sending only inputs smaller than \< 512.

Additionally, you could send in the `parameters` key of your request configuration to automatically truncate any incoming sequence, meaning the inference pipeline would automatically cut after 512 tokens.

```python
long_sentence = "...." # longer than 512 tokens
sentiment_input= {
   {'inputs':long_sentence,
    'parameters': {'truncation':True}
   }
predictor.predict(sentiment_input)

```

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-are-the-inputs-tokenized-when-model-deployment%2F9692)._