Karna to Firefox@lemmy.ml • 2 years agoMeet Orbit, Mozilla's AI Assistant Extension for Firefoxwww.omgubuntu.co.ukexternal-linkmessage-square54fedilinkarrow-up1133arrow-down153cross-posted to: news
arrow-up180arrow-down1external-linkMeet Orbit, Mozilla's AI Assistant Extension for Firefoxwww.omgubuntu.co.ukKarna to Firefox@lemmy.ml • 2 years agomessage-square54fedilinkcross-posted to: news
minus-squareKarnaOPlinkfedilink21•2 years agoIn such scenario you need to host your choice of LLM locally.
minus-square@ReversalHatchery@beehaw.orglinkfedilinkEnglish5•2 years agodoes the addon support usage like that?
minus-squareKarnaOPlinkfedilink7•2 years agoNo, but the “AI” option available on Mozilla Lab tab in settings allows you to integrate with self-hosted LLM. I have this setup running for a while now.
minus-square@cmgvd3lw@discuss.tchncs.delinkfedilink4•2 years agoWhich model you are running? Who much ram?
minus-squareKarnaOPlinkfedilink4•edit-22 years agoMy (docker based) configuration: Software stack: Linux > Docker Container > Nvidia Runtime > Open WebUI > Ollama > Llama 3.1 Hardware: i5-13600K, Nvidia 3070 ti (8GB), 32 GB RAM Docker: https://docs.docker.com/engine/install/ Nvidia Runtime for docker: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html Open WebUI: https://docs.openwebui.com/ Ollama: https://hub.docker.com/r/ollama/ollama
In such scenario you need to host your choice of LLM locally.
does the addon support usage like that?
No, but the “AI” option available on Mozilla Lab tab in settings allows you to integrate with self-hosted LLM.
I have this setup running for a while now.
Which model you are running? Who much ram?
My (docker based) configuration:
Software stack: Linux > Docker Container > Nvidia Runtime > Open WebUI > Ollama > Llama 3.1
Hardware: i5-13600K, Nvidia 3070 ti (8GB), 32 GB RAM
Docker: https://docs.docker.com/engine/install/
Nvidia Runtime for docker: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
Open WebUI: https://docs.openwebui.com/
Ollama: https://hub.docker.com/r/ollama/ollama