datasets<\/em> for loading and processing datasets.<\/p>\n!pip install \"transformers\"\n!pip install \"torch\"\n!pip install \"datasets\"\n!pip install \"peft\"<\/pre>\nLoad model and tokenizer<\/strong><\/p>\nThe script loads the base model and tokenizer for the Llama model from Hugging Face Transformers using the LlamaForCausalLM<\/strong><\/em> and LlamaTokenizer<\/em><\/strong> classes.<\/p>\nmodel = LlamaForCausalLM.from_pretrained(\n base_model,\n # load_in_8bit=True, # Add this for using int8\n torch_dtype=torch.float16,\n device_map=device_map,\n )\n\u200b\n tokenizer = LlamaTokenizer.from_pretrained(base_model)\n tokenizer.pad_token_id = 0\n<\/pre>\nLoad data<\/strong><\/p>\nDataset loading is based on the file format specified by the data_path argument. load a dataset either from a JSON or JSON Lines file (specified by the data_path argument as a file path) or from a dataset available in the Hugging Face datasets library (specified by the data_path argument as a dataset name). The loaded dataset is stored in the data variable, which can then be further processed and tokenized for model training.<\/p>\n
if data_path.endswith(\".json\") or data_path.endswith(\".jsonl\"):\n data = load_dataset(\"json\", data_files=data_path)\n else:\n data = load_dataset(data_path)\n<\/pre>\nTokenize data<\/strong><\/p>\nThe tokenize()<\/strong><\/em> function is defined to preprocess the data. It encodes the input and output text data into token IDs, concatenates them, and appends an end-of-sequence token. The encoded data is then prepared as input for the model. The data is split into training and validation sets using the train_test_split()<\/em><\/strong> function from the datasets library.<\/p>\ndef tokenize(data):\n source_ids = tokenizer.encode(data['input'])\n target_ids = tokenizer.encode(data['output'])\n\u200b\n input_ids = source_ids + target_ids + [tokenizer.eos_token_id]\n labels = [-100] * len(source_ids) + target_ids + [tokenizer.eos_token_id]\n\u200b\n return {\n \"input_ids\": input_ids,\n \"labels\": labels\n }\n\u200b\n#split thte data to train\/val set\ntrain_val = data[\"train\"].train_test_split(\n test_size=val_set_size, shuffle=False, seed=42\n)\ntrain_data = (\n train_val[\"train\"].shuffle().map(tokenize)\n)\nval_data = (\n train_val[\"test\"].shuffle().map(tokenize)\n \n)\n<\/pre>\ninput_ids<\/em><\/strong> are created by concatenating the token IDs of the source and target text, followed by the EOS token ID. This forms the input sequence for the language model during training. Given the input, the model will be trained to predict the target sequence.<\/p>\nlabels<\/em><\/strong> are created by concatenating the list of -100 values (for the source text), followed by the token IDs of the target text, and finally, the EOS token ID. This labels list serves as the target labels for the language model during training. The model will be trained to predict the target sequence (token IDs in the labels) from the input sequence (token IDs in the input_ids).<\/p>\n[-100] * len(source_ids)<\/strong>: This creates a list of -100 values with a length equal to the number of tokens in the source text. The value -100 is a special value used to mask tokens that do not require predicting (i.e., padding tokens).<\/p>\nInitiate the trainer<\/strong><\/p>\nThe script creates a transformers.Trainer object<\/em><\/strong> for training the model. It uses the train_data<\/em><\/strong> and val_data<\/em><\/strong> datasets and various training configurations such as batch size, gradient accumulation steps, number of epochs, learning rate, etc.<\/p>\n trainer = transformers.Trainer(\n model=model,\n train_dataset=train_data,\n eval_dataset=val_data,\n args=transformers.TrainingArguments(\n per_device_train_batch_size=micro_batch_size,\n gradient_accumulation_steps=gradient_accumulation_steps,\n warmup_steps=100,\n num_train_epochs=num_epochs,\n learning_rate=learning_rate,\n fp16=True,\n logging_steps=10,\n optim=\"adamw_torch\",\n evaluation_strategy=\"steps\",\n save_strategy=\"steps\",\n eval_steps=200,\n save_steps=200,\n output_dir=output_dir,\n save_total_limit=3\n ),\n data_collator=transformers.DataCollatorForSeq2Seq(\n tokenizer, pad_to_multiple_of=8, return_tensors=\"pt\", padding=True\n ),\n )\n\u200b\n trainer.train()<\/pre>\nSave Model<\/strong><\/p>\nAfter training, the final model is saved to the specified output_dir<\/em><\/strong>. <\/p>\n model.save_pretrained(output_dir)\n<\/pre>\nRun Model<\/strong><\/p>\npython train.py --base_model meta-llama\/Llama-2-7b --data_path tatsu-lab\/alpaca --output_dir output\/<\/pre>\nHow to fine-tune Llama2 using LORA<\/strong><\/h2>\nHere, I explain the steps to fine-tune LLaMA 2 in this example using Low-Rank Adaptation (LoRA). LoRA proposes to freeze pre-trained model weights and inject trainable layers (rank-decomposition matrices) in each transformer block. This greatly reduces the number of trainable parameters and GPU memory requirements since gradients don\u2019t need to be computed for most model weights.<\/p>\n
The steps to fine-tune LLaMA 2 using LoRA is the same as of SFT. In the code, when loading the model and tokenizer, you need to specify the LoRA parameters. A sample code for fine-tuning LLaMA2 with LoRA is provided below. <\/p>\n
from typing import List\n\u200b\nimport fire\nimport torch\nimport transformers\nfrom datasets import load_dataset\n\u200b\n\u200b\nfrom peft import (\n LoraConfig,\n get_peft_model,\n get_peft_model_state_dict,\n prepare_model_for_int8_training,\n set_peft_model_state_dict,\n PrefixTuningConfig,\n TaskType\n)\nfrom transformers import LlamaForCausalLM, LlamaTokenizer\n\u200b\n\u200b\n\u200b\ndef train(\n # model\/data params\n base_model: str = \"\", \n data_path: str = \"\",\n output_dir: str = \"\",\n\u200b\n micro_batch_size: int = 4,\n gradient_accumulation_steps: int = 4,\n num_epochs: int = 3,\n learning_rate: float = 3e-4,\n val_set_size: int = 2000,\n \n # lora hyperparams\n lora_r: int = 8,\n lora_alpha: int = 16,\n lora_dropout: float = 0.05,\n lora_target_modules: List[str] = [\n \"q_proj\",\n \"v_proj\",\n ]\n):\n\u200b\n device_map = \"auto\"\n\u200b\n\u200b\n # Step 1: Load the model and tokenizer\n\u200b\n model = LlamaForCausalLM.from_pretrained(\n base_model,\n # load_in_8bit=True, # Add this for using int8\n torch_dtype=torch.float16,\n device_map=device_map,\n )\n\u200b\n tokenizer = LlamaTokenizer.from_pretrained(base_model)\n tokenizer.pad_token_id = 0\n\u200b\n Add this for training LoRA\n\u200b\n config = LoraConfig(\n r=lora_r,\n lora_alpha=lora_alpha,\n target_modules=lora_target_modules,\n lora_dropout=lora_dropout,\n bias=\"none\",\n task_type=\"CAUSAL_LM\",\n )\n model = get_peft_model(model, config)\n\u200b\n model = prepare_model_for_int8_training(model) # Add this for using int8\n\u200b\n\u200b\n # Step 2: Load the data\n\u200b\n if data_path.endswith(\".json\") or data_path.endswith(\".jsonl\"):\n data = load_dataset(\"json\", data_files=data_path)\n else:\n data = load_dataset(data_path)\n \n # Step 3: Tokenize the data\n\u200b\n def tokenize(data):\n source_ids = tokenizer.encode(data['input'])\n target_ids = tokenizer.encode(data['output'])\n\u200b\n input_ids = source_ids + target_ids + [tokenizer.eos_token_id]\n labels = [-100] * len(source_ids) + target_ids + [tokenizer.eos_token_id]\n\u200b\n return {\n \"input_ids\": input_ids,\n \"labels\": labels\n }\n\u200b\n #split thte data to train\/val set\n train_val = data[\"train\"].train_test_split(\n test_size=val_set_size, shuffle=False, seed=42\n )\n train_data = (\n train_val[\"train\"].shuffle().map(tokenize)\n )\n val_data = (\n train_val[\"test\"].shuffle().map(tokenize)\n \n )\n\u200b\n # Step 4: Initiate the trainer\n\u200b\n trainer = transformers.Trainer(\n model=model,\n train_dataset=train_data,\n eval_dataset=val_data,\n args=transformers.TrainingArguments(\n per_device_train_batch_size=micro_batch_size,\n gradient_accumulation_steps=gradient_accumulation_steps,\n warmup_steps=100,\n num_train_epochs=num_epochs,\n learning_rate=learning_rate,\n fp16=True,\n logging_steps=10,\n optim=\"adamw_torch\",\n evaluation_strategy=\"steps\",\n save_strategy=\"steps\",\n eval_steps=200,\n save_steps=200,\n output_dir=output_dir,\n save_total_limit=3\n ),\n data_collator=transformers.DataCollatorForSeq2Seq(\n tokenizer, pad_to_multiple_of=8, return_tensors=\"pt\", padding=True\n ),\n )\n\u200b\n trainer.train()\n\u200b\n\u200b\n # Step 5: save the model\n model.save_pretrained(output_dir)\n\u200b\n\u200b\n\u200b\nif __name__ == \"__main__\":\n fire.Fire(train)\n\n<\/pre>\nLoraConfig<\/em><\/strong> is a class provided by the PeFT framework that allows you to configure the LoRA training process. The code shows how to create a LoraConfig<\/em><\/strong> object with the following parameters;<\/p>\nr<\/em><\/strong>: the rank of the update matrices, expressed in int. Lower rank results in smaller update matrices with fewer trainable parameters.
lora_alpha<\/em><\/strong>: LoRA scaling factor
target_modules<\/em><\/strong>: The modules (for example, attention blocks) to apply the LoRA update matrices.
bias<\/em><\/strong>: Specifies if the bias parameters should be trained. Can be \u2018none\u2019, \u2018all\u2019 or \u2018lora_only\u2019.<\/p>\nThe code further calls the get_peft_model()<\/em><\/strong> function to create a PeFT model by integrating the LoRA configurations into the base model. The result is assigned back to the model variable. The prepare_model_for_int8_training()<\/em><\/strong> function is applied to the model to enable int8 quantization. This function is part of the PeFT framework, and it prepares the model to be trained using 8-bit precision, which can help reduce the model\u2019s memory and computational footprint. int8<\/em> quantization is an optimization technique and can be used to reduce memory usage during inference but does not affect the actual training process itself.<\/p>\n<\/p><\/div>\n
\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"LLaMA 2 is finally here, and it doesn\u2019t disappoint. The new model has substantial performance…<\/p>\n","protected":false},"author":1,"featured_media":1098,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1794,2205],"tags":[],"class_list":["post-1097","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-llm"],"_links":{"self":[{"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/posts\/1097","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/comments?post=1097"}],"version-history":[{"count":0,"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/posts\/1097\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/media\/1098"}],"wp:attachment":[{"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/media?parent=1097"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/categories?post=1097"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/x.sheep-mine.ts.net\/index.php\/wp-json\/wp\/v2\/tags?post=1097"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"https:\/\/blog.accubits.com\/wp-content\/uploads\/2023\/07\/LLAMA2-benchmark-1024x424.png\")
<\/figure>\n<\/div>\n![\"\"](\"https:\/\/blog.accubits.com\/wp-content\/uploads\/2023\/07\/LLAMA2-benchmark-againt-closed-1024x274.png\")
<\/figure>\n<\/div>\n