How does Prefix Tuning work?

The prefix typically consists of a few hundred trainable vectors per layer, totaling 0.1% to 1% of base model parameters. Prefix tuning is conceptually similar to soft prompts but operates at every attention layer rather than just the input embedding layer, giving it more representational capacity per trainable parameter. The platform strengthens ente