Why "element 0 of tensors does not require grad" Shows Up in LoRA Fine-Tuning

Is this still a bug?

It was a real and very common one, and for the most part it has been fixed in the libraries themselves. When you call get_peft_model, PEFT runs a preparation step for gradient checkpointing: for a non-quantized model it calls enable_input_require_grads if the model supports it, and otherwise registers a forward hook on the input embeddings that forces their output to require grad. Either way, the input that gradient checkpointing cares about now requires grad, and the backward graph is built.

For a long time this preparation only covered one ordering, enabling gradient checkpointing before wrapping the model with get_peft_model. The reverse, enabling it afterward, would silently leave the loss detached without raising a warning, which is its own kind of trap. That remaining gap was closed in early 2025 (PEFT pull request #2398), so on a current install both orderings work. Indeed, when the maintainers tried to reproduce the failure by moving the gradient checkpointing call after get_peft_model, they found it no longer triggered without going out of their way. If your minimal script runs cleanly, this is why.

from transformers import AutoModelForCausalLM
from peft import LoraConfig, get_peft_model

model = AutoModelForCausalLM.from_pretrained("your-model-id")
model.gradient_checkpointing_enable()          # save memory by recomputing activations
model.config.use_cache = False                 # required alongside gradient checkpointing

lora_config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
model = get_peft_model(model, lora_config)

# on a current peft/transformers this runs fine: get_peft_model has already
# prepared the model for gradient checkpointing.
# on older versions, or in the situations listed below, the backward pass raises:
#   UserWarning: None of the inputs have requires_grad=True. Gradients will be None
#   RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Why it happens

Gradient checkpointing saves memory by not storing a block's intermediate activations and recomputing them during the backward pass. The default implementation is reentrant, and it decides whether to track that recomputation for gradients by looking at whether the block's inputs require grad. With LoRA the base model is frozen, so the first input, the output of the frozen token embeddings, does not require grad. The reentrant checkpoint therefore builds no backward graph through the block, even though the LoRA parameters inside it do require grad. With no path back to them, the loss comes out with requires_grad set to False, and calling backward on it raises the error. The trainable parameters were there the whole time; the graph that reaches them was missing.

Put another way, the automatic fix and the manual ones all do the same thing: they make sure at least one input into the checkpointed region requires grad, so the reentrant checkpoint builds the graph. That is the whole of it.

When you still see it

The automatic preparation covers the common path, but it does not cover every path. You are still likely to run into the error in one of these situations:

• Older, pinned versions of PEFT or Transformers. A great deal of tutorial code and many repositories sit on versions from before the preparation step covered your case, and they reproduce the error exactly as the original issues described.
• Custom training loops or model wrappers. If your code never goes through get_peft_model's preparation, or wraps the model in a way that hides the embeddings from it, the hook is never attached.
• Models whose embeddings the fix cannot reach. On some multimodal or custom architectures, enable_input_require_grads or get_input_embeddings is not implemented and raises NotImplementedError, so the automatic hook never gets registered.
• Quantized setups that skip the helper. For QLoRA, prepare_model_for_kbit_training performs the same preparation; without it, gradient checkpointing on a 4-bit model lands you back on the error.

The one-line diagnostic

Whichever situation you are in, you can confirm it in one line before launching a full run. Do a forward pass on a batch and check whether the loss requires grad. If it is False, the backward graph is broken and training will fail the moment it starts.

out = model(**batch)            # one forward pass on a training batch
print(out.loss.requires_grad)   # False means nothing will backpropagate

The fixes

The simplest fix is the one many readers apply without realizing it: upgrade PEFT and Transformers, and let get_peft_model do the preparation. When that is not an option, or when your model is one the automatic path cannot reach, the manual fixes are the same ones PEFT applies internally. The first makes the block's input require grad directly. enable_input_require_grads registers a hook on the embeddings that sets their output to require grad.

model = AutoModelForCausalLM.from_pretrained("your-model-id")
model.gradient_checkpointing_enable()
model.enable_input_require_grads()             # make the embedding output require grad
model.config.use_cache = False
model = get_peft_model(model, lora_config)

The second avoids the reentrant implementation altogether. The non-reentrant checkpoint tracks tensors that require grad regardless of whether the block inputs do, so it does not have the limitation in the first place.

# alternative: the non-reentrant checkpoint does not have this limitation
model.gradient_checkpointing_enable(gradient_checkpointing_kwargs={"use_reentrant": False})

# or, through the Trainer:
from transformers import TrainingArguments
args = TrainingArguments(
    output_dir="out",
    gradient_checkpointing=True,
    gradient_checkpointing_kwargs={"use_reentrant": False},
)

For 4-bit QLoRA the preparation lives in prepare_model_for_kbit_training, which calls enable_input_require_grads for you, along with other setup. If you skip that helper and enable gradient checkpointing by hand, you hit the same error, so either use the helper or add the line yourself.

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import prepare_model_for_kbit_training, get_peft_model

bnb = BitsAndBytesConfig(load_in_4bit=True)
model = AutoModelForCausalLM.from_pretrained("your-model-id", quantization_config=bnb)
model = prepare_model_for_kbit_training(model)   # calls enable_input_require_grads() for you
model = get_peft_model(model, lora_config)

Verifying the fix

After any of these, the loss requires grad again and training proceeds. print_trainable_parameters is a good second check that the adapters are attached and are the only trainable parameters.

out = model(**batch)
print(out.loss.requires_grad)        # now True
model.print_trainable_parameters()   # confirms the adapters are the trainable params

Common questions

Is the "element 0 of tensors does not require grad" error still a problem?

For the common LoRA path, mostly not. Recent PEFT versions prepare the model for gradient checkpointing inside get_peft_model, so a standard script no longer raises it. The error now tends to come from older versions, custom training code, models whose embeddings the automatic hook cannot reach, or quantized setups that skip prepare_model_for_kbit_training.

Why does my LoRA plus gradient checkpointing script not error anymore?

Because get_peft_model now does the work the error used to force on you. It calls enable_input_require_grads, or registers a forward hook on the input embeddings, so their output requires grad and the reentrant checkpoint can build the backward graph. The fix you would have added by hand is already applied.

When do I still need enable_input_require_grads?

When the automatic preparation does not run or does not reach your model: an older PEFT version, a custom loop that bypasses get_peft_model, or an architecture where get_input_embeddings is not implemented. For 4-bit QLoRA, prepare_model_for_kbit_training covers it; if you set things up by hand and skip that helper, add the line yourself, or switch to the non-reentrant checkpoint.