LoRA Finetuning

  1. Merging multiple LoRAs: orthogonal weights, masked dimensions

  2. Decomposition