The main way most folks install and run Ollama is to start with the installer that you can find on ollama.com. You can be up and running super quickly with that. But some like to use Docker. Docker is nice because it is fully self-contained. For programs that are messy with a lot of dependencies, it can be nice to use Docker because when you delete the container and image, there is nothing left. Ollama is a very simple program with a single executable, but still its convenient for some to have everything running in Docker. You will take a little performance hit, though. On Linux, it's pretty minimal, but on Mac and Windows it will be a little more significant due to the virtual machine that needs to run underneath Docker. Also, on Mac, there is no GPU pass-through possible, due to the Docker platform not supporting it. Just on Mac.

Once you know you want to run Ollama in Docker, you still need to put your models in a directory on your base system. Typically, docker containers and images are designed to be really small. But the models you run with Ollama are enormous. So that's why they are separate. Though, with Docker, it may be touch easier to put your models anywhere you like. 

I am going to assume that Docker is already set up on your machine and is ready to go. The command to run is `docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama`. Run it, and then we can take a look at the command in a bit more detail. `docker run` is the command to start a new container from an image. If you don't have the image, it will be pulled from Docker Hub. `-d` says to run the container as a daemon, so it's running in the background. `--gpus=all` allows for GPU pass-through. BUT, this doesn't work on a Mac. And on Linux and Windows, you need to make sure the nvidia toolkit is installed, assuming you have an nvidia card. I am running this on a brev.dev instance which already has that installed, but you can find the instructions to install it in the README on the DockerHub page for the Ollama image. I'll talk more about Brev in another video coming very soon. 

The next part of the command is the volume mount. This is saying create a volume named ollama and mount it to /root/.ollama inside the container. This is a little confusing as there are different ways to use volume mounts. But basically, there are two sides of any command that has a colon. The left side is usually what's on the host system and the right side is what is in the container. If the left side is a word and not a path, then the Docker process will create a new directory where its default is. On this system that is /var/lib/docker/volumes and there you can see an ollama directory. But if you want the models to go somewhere else, you need to specify the fully qualified path before the colon. I am currently in /home/ubuntu and if I want models to go into the myollamamodels directory which I have already created, I would have to use -v /home/ubuntu/myollamamodels:/root/.ollama. 

Next comes ports. Any time there is a port that you need to access inside the container and want to use it outside, you have to declare it. So we want port 11434 inside the container to be available outside the container as well. After that, we set up a name. If you don't include a name, one will be generated. So its usually best to assign one that you like. Finally we see the Docker image being used: ollama/ollama which corresponds to the url on Dockerhub which is https://hub.docker.com/r/ollama/ollama. 

Let's say you actually want to call the container something else, or you want to move the container to a new location, or there is a new version and you need to update the ollama container. Well in all of those cases, if you just run the command again it will fail, because there is a running container already. So we need to stop it and remove it. Luckily we set a good name so this is pretty easy. Just run `docker stop ollama` and then `docker rm ollama`. Now if you need a new ollama image, you can run `docker pull ollama/ollama` to grab the latest. Then you can run that `docker run` command again with the correct parameters. One thing that confuses new users is that they want to update the software inside the container, instead of pulling a new image. Images and containers are designed to be immutable. You don't want to update the contents of the container. If there is updated software, you pull the new version of the image and start that up. Updating the software inside a container will probably make it break much more easily. Don't do it. 

Of course you didn't run ollama just to run the server, you probably want to use the client to ask a question. But ollama probably isn't installed on your host system. To run the ollama client that is inside the container, we need to use the `docker exec` command. `docker exec -it ollama ollama run llama2`. So that's `docker exec` which will run a command inside a container. `-it` says to run it in an interactive terminal. If you just needed to run the command and exit, you could leave that off. The first `ollama` tells docker exec which container to use. If you hadn't specified a name when running the image, then you would have to specify a container id which is a unique string each time it's run. Everything after the container name is the command to run on the host. So `ollama run llama2`. 

Now I have used the terms container and image a lot, and if you aren't familiar, you might think I am using them interchangeably. But they are different. All the things on DockerHub are called images. ollama/ollama is an image. It defines how ollama should run. A container is a lightweight method to run an image. You can think of the container as the instantiation of the image. And we can see this on our system. We stopped and removed the container just now, but the image still existed and we can quickly run it. If you run `docker images` you will see a list of all the images you have downloaded to run as containers. If you want to remove an image, then the command is `docker rmi ` and the image name but it will only delete if there are no containers currently using the image. 

OK, back to ollama run. We were looking at `docker exec -it ollama ollama run llama2`. If you run that you will be dropped into the interactive repl to work with ollama. If you are going to be doing this a lot, then it probably makes sense to make an alias. On linux and mac, assuming you are using the bash or zsh shell, the command is `alias ollama="docker exec -it ollama ollama`. If using fish, then just replace that equals sign with a space. If you are a Windows user, you probably know the right command to create an alias like that. 

Now you can run `ollama run llama2` just like you do on systems with ollama installed. If you want to use that alias after this terminal session is over, you will want to add the command to one of your rc files for your shell. That's going to be a file like .bashrc or .profile or .zshrc or config/fish/config.fish.

There is another command you need to know about. Sometimes you have to look at the logs to see what ollama is doing. So `docker logs ollama` is the command to run to see those. 

ok, so now you are setup to use ollama in docker on a host. But let's say you have setup ollama on a host, like I have on brev, and you want to access it from a remote machine, something other than the host running docker. Well if the host is on the same network as your client machine, this is easy. You will have to stop and remove your docker container for ollama. Remember, that was `docker stop` and `docker rm`. Then run the same run command but add `OLLAMA_HOST=0.0.0.0:11434` before docker run on the same line. And then anything that connects to the host on that port will see your docker container version of ollama.

But what happens if that host isn't on your home network. Instead its somewhere else on the Internet. Well then things get a bit tougher. Somehow you need to give access to the host to your client. Some will solve this by opening ports, but that's an incredibly stupid thing to do without blocking access with some sort of authentication. Probably the easiest thing to do is to use tailscale, which is free for most non-companies. For you at home, this is brain dead easy. And if you are doing this for work, then your ops team probably has a solution they like or tailscale is so ridiculously cheap that you can probably expense it and they wont bat an eyelash over it. 

Side note, I once bought some stuff for the office when I was at datadog. It was the only time finance reached out and they said I needed to describe what I bought. 'You really don't want to know' I said. But I had to put it in. new people , new system. Don't worry they said. OK, so I put in what I bought. About $200 worth of NERF guns and those little yellow balls for ammo. I thought, oh well, guess I'll eat that expense.  No problem. expense paid. Unfortunately the cleaning staff thought the balls were trash and within a week or so, all the ammo was gone. 

Anyway, that's what you need to know to understand how to work with Docker and Ollama. In most cases, you will want to run Ollama as a native install, but if you insist on using docker, now you are set. If you have any questions, let me know in the chat below. I love the questions that I see, especially when you give me suggestions for what to make a video about. So many of these videos, including this one, come from those suggestions. Keep them coming. 

Thanks so much for being here, goodbye. 

