[{"data":1,"prerenderedAt":107},["ShallowReactive",2],{"glossary-en-model-training-optimization":3},{"id":4,"title":5,"body":6,"description":96,"extension":97,"meta":98,"navigation":102,"path":103,"seo":104,"stem":105,"__hash__":106},"en_glossary/en/glossary/model-training-optimization.md","Model Training and Optimization: Pre-training, Fine-tuning, LoRA, Quantization",{"type":7,"value":8,"toc":85},"minimark",[9,14,18,23,26,30,33,37,40,44,47,52,67],[10,11,13],"h2",{"id":12},"model-training-and-optimization","Model Training and Optimization",[15,16,17],"p",{},"These terms describe the journey of a model from a raw state to a capable assistant, and how we make it efficient enough to run on various hardware.",[19,20,22],"h3",{"id":21},"pre-training","Pre-training",[15,24,25],{},"The initial stage where a model learns basic language capabilities and general knowledge from a massive dataset (almost the entire internet). It's like teaching a child to read and providing them with a general encyclopedia.",[19,27,29],{"id":28},"fine-tuning","Fine-tuning",[15,31,32],{},"The process of taking a pre-trained model and training it further on a smaller, specific dataset to improve performance in a specific task (e.g., medical advice or code generation).",[19,34,36],{"id":35},"lora-low-rank-adaptation","LoRA (Low-Rank Adaptation)",[15,38,39],{},"A technique that allows Fine-tuning to be performed with much less computational power. Instead of updating all parameters, LoRA updates only a small, specific part of the model network.",[19,41,43],{"id":42},"quantization","Quantization",[15,45,46],{},"The process of compressing a model's weights (parameters) to take up less space. For example, reducing 16-bit data to 4-bit allows the model to consume specific RAM.",[48,49,51],"h4",{"id":50},"industrial-relevance","Industrial Relevance",[15,53,54,55,58,59,62,63,66],{},"Techniques like ",[56,57,43],"strong",{}," and ",[56,60,61],{},"LoRA"," are critical for ",[56,64,65],{},"Edge AI",". They make it possible to run powerful models on constrained hardware.",[68,69,70],"ul",{},[71,72,73,76,77,84],"li",{},[56,74,75],{},"Potential Application",": In the future, advanced devices similar to the ",[56,78,79],{},[80,81,83],"a",{"href":82},"/en/products/zma-data-acquisition","ZMA Data Acquisition"," could use quantized models to perform local anomaly detection without needing a constant cloud connection.",{"title":86,"searchDepth":87,"depth":87,"links":88},"",2,[89],{"id":12,"depth":87,"text":13,"children":90},[91,93,94,95],{"id":21,"depth":92,"text":22},3,{"id":28,"depth":92,"text":29},{"id":35,"depth":92,"text":36},{"id":42,"depth":92,"text":43},"Key concepts in creating efficient AI models: From Pre-training on massive data to optimizing for edge devices with Quantization.","md",{"tags":99},[100,101,29,61,43,65],"AI","Model Training",true,"/en/glossary/model-training-optimization",{"title":5,"description":96},"en/glossary/model-training-optimization","vRcdhMya07FZJHMH175LXpY8D5JtGLwoe3kBNeQLCMM",1778229654821]