tom
11/29/2024, 1:44 PMflatpak install flathub com.jeffser.Alpaca
- Much faster than brew package!
- also maintains ollama version to
Cons:
- Can only interact with the models from their simplistic (but powerful) GUI, not possible through CLI
- This is not desirable for my use cases!
3) Using official install instructions: curl -fsSL https://ollama.com/install.sh | sh
Pros:
- Official method which supposedly also sets up systemd services
- Enables CLI use
Cons:
- Install does not work on Fedora Atomic running from CLI
> Question HERE! Any thoughts on how to install ollama through this method? Perhaps from the Bluebuild side?
4) Inside a docker container:
Pros: N/A
Cons: Extremely clunky to set up and maintain...
Any thoughts on how to make #3 work? How are other users using LLMs locally?
ThanksAnswer Overflow
11/29/2024, 1:44 PMfiftydinar
11/29/2024, 3:40 PMfiftydinar
11/29/2024, 3:41 PMnvidia-smi works in build-timefiftydinar
11/29/2024, 3:41 PMfiftydinar
11/29/2024, 3:42 PMxyny
11/29/2024, 3:55 PMxyny
11/29/2024, 3:56 PMxyny
11/29/2024, 3:58 PMflatpak run --command might work for thatxyny
11/29/2024, 3:58 PMxyny
11/29/2024, 4:01 PMujust ollama command on Bluefin, so looking at the source code of that would probably helpxyny
11/29/2024, 4:01 PMVirt-io
11/29/2024, 8:27 PM[Pod]
Network=bridge
NetworkAlias=llm_network
PodName=llm_pod
# ollama
PublishPort=127.0.0.1:11434:11434
# open-webui
PublishPort=127.0.0.1:3000:8080
ollama-app.container
[Unit]
Description=Ollama
[Container]
Label=app=ollama
ContainerName=ollama
Pod=llm.pod
Image=docker.io/ollama/ollama:latest
AddDevice=nvidia.com/gpu=all
SecurityLabelDisable=true
Environment=OLLAMA_FLASH_ATTENTION=1
Environment=OLLAMA_NOHISTORY=1
Volume=/home/username/Documents/Containers/Ollama:/root/.ollama:Z
[Install]
WantedBy=multi-user.target default.target
openwebui-app.container
[Unit]
Description=Open-WebUI
[Container]
Label=app=open-webui
ContainerName=open-webui
Pod=llm.pod
Image=ghcr.io/open-webui/open-webui:main
Environment=OLLAMA_BASE_URL=http://127.0.0.1:11434
Volume=/home/username/Documents/Containers/Open-webui:/app/backend/data:Z
[Install]
WantedBy=multi-user.target default.targettom
11/30/2024, 11:00 AMtom
11/30/2024, 11:01 AMflatpak run --command=ollama com.jeffser.Alpaca serve
and then from another terminal window:
flatpak run --command=ollama com.jeffser.Alpaca list
but strangely this internal ollama server is NOT the same as the one used by the alpaca GUI - it's a separate instance. This meanst hat you the models you pull from alpaca gui do not appear in this ollama instance, and the reverse is also true...
Alpaca makes it dead simple to access ollama. I mean on paper it's basically the same as setting up a docker/distrobox container manually. This is indeed a good alternative in terms of simplicity.. but not the best way for power users.tom
11/30/2024, 11:06 AMscripts module approach. The script will draw from the official ollama install.sh script, along with with a usermod script and systemd service file that also gets created. Drawing maximum inspiration from the SDDM method shown by wayblue:
https://github.com/wayblueorg/wayblue/blob/live/files/systemd/system/sddm-boot.service
https://github.com/wayblueorg/wayblue/blob/live/files/system/etc/sddm/sddm-useradd
Should get to that next week. Will keep you in the loop, and close this issue when I get it working.tom
12/05/2024, 11:06 AMfiftydinar
12/05/2024, 3:20 PMtom
12/05/2024, 3:23 PMfiftydinar
12/05/2024, 3:33 PM