<!-- SC_OFF --><div class="md"><p>Not everyone has the disposable income to build a small data center, so making this post for the underdogs as I was very surprised by the performance/results of this 35B MOE model. Context is admittedly tight and will get laughed at by the big bo…
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1vsx94f/we_quantized_the_new_ornith_15_9b_and_35ba3b/"> <img alt="We quantized the new Ornith 1.5 9B and 35B-A3B" src="https://preview.redd.it/p5pjiui20ekh1.png?width=140&height=111&auto=webp&s=855…
<!-- SC_OFF --><div class="md"><p>(my own experience)</p> <p>I have a mid-range gaming laptop from 2024 with 4GB VRAM and 16GB RAM. There is nothing that I can run other than small models at humble t/s. </p> <p>I mostly use the models to generate scripts for medical physics resea…