I will present a practical workflow for performing supervised fine-tuning of a small language model (such as SmolLM2-360M-Instruct) on an ordinary laptop (CPU-only, 16GB RAM). The talk covers instruction dataset formatting, supervised fine-tuning, and inference. A brief before-and-after comparison will show the effect of the fine-tuning. In particular:
(1) instruction dataset formatting - The dataset will be a jsonl (One JSON object per line) file, and each line contains the mapping between requirement and python function (that is, the function name and its arguments). Each line is in the following format:
===========
{“instruction”: , "input": , "output": "<|begin|>\n### Requirement:\n\n### Parsed:\n<function name, actual argument name and value>\n### Python Code:\n<function_name(argument_name=argument_value, ...)>\n<|end|>"} =========== (2) supervised fine-tuning - the training pipeline contains the following steps: ===========

1. Handle commandline arguments – argparse.ArgumentParser()
2. Load tokenizer – tokenizer = AutoTokenizer.from_pretrained(…)
3. Load model – model = AutoModelForCausalLM.from_pretrained(…, device_map=”cpu”, low_cpu_mem_usage=True)
4. Prepare model for PEFT/LoRA using get_peft_model(model, peft.LoraConfig(…))
5. Load dataset (JSONL with ‘instruction’ and ‘response’ keys) – dataset = load_dataset(…)
6. Tokenize dataset and create labels with prompt masking – tokenized = dataset.map(…) and customized functions
7. Convert to torch tensors (because the trainer expects tensors) – transformers.DataCollatorForLanguageModeling(…)
8. Setup TrainingArguments using training_args = TrainingArguments(…)
9. Run training ( ~20 minutes “per epoch” for a small instruction dataset with 100 samples on a CPU-only average laptop) – transformers.Trainer.train()(…)
10. Save the LoRA-based fine-tuned model (weights and configuration) – model.save_pretrained(…)
11. Save the tokenizer and tokenization configuration (things needed to turn text into model input) – tokenizer.save_pretrained(…)

===========

(3) inference and evaluation - the inference pipeline contains the following steps:

1. Load tokenizer – AutoTokenizer.from_pretrained(…)
2. Load base model – AutoModelForCausalLM.from_pretrained(…, torch_dtype=torch.float16, device_map=”auto”)
3. Attach LoRA adapter – model = PeftModel.from_pretrained(base_model, args.output_dir)

4. Generate the formatted prompt – prompt = “<

   begin

   >\n” + + “\n”

5. Tokenize the input – inputs = tokenizer(prompt, return_tensors=”pt”).to(model.device)
6. Perform inference – outputs = model.generate(…)
7. Generate the text response – response = tokenizer.decode(outputs[0], skip_special_tokens=True)

===========