technovangelist.videoprojects/scripts/ollamaandlangchaininpython.txt

Can you talk to your own documents to get answers out of them? That’s the goal for many tools out there, and the concept is called embeddings. Feed your notes, your collected pdfs, and anything else you have, to your LLM and ask it questions to generate new insights. Ollama is already the easiest way to work with LLMs on your laptop, but can we support embeddings? In this video, I'll show you how a Python developer can use embeddings on Ollama using LangChain. 

So let's say you have a document that you want to ask a question about. I am going to use this Wikipedia article about the horrible fires on Maui just a few days ago. The URL is on the screen. So how do you get started?

Since we are using LangChain in this video, lets start with 'pip install langchain'. OK, now create a python file and start building. we first need to import Ollama. 'from langchain.llms import ollama'. Then we need to initiate our connection to a model on ollama. 'ollama = ollama base url = http :// localhost : 11434 and model = llama2'. we can specify any model here that either is on the ollama hub or that you have created locally. Now to test that that works, 'print ollama why is the sky blue'.  Now if we run that, we will see why the sky is blue. perfect. 

Now we can work on loading the document. When I did this the first time, i needed the python module bs4, so run 'pip install bs4'. Now we can add the import for the loader: 'from langchain.document loaders import webbaseloader'. then load the document. 'loader = webbaseloader and the url,  then data = loader.load'. 

Now when we ask it a question, chances are the whole document isn't needed to answer the question, but rather just the parts that talk about the concept I am interested in. But how do we limit what gets sent to the model. Databases are great for searching for a subset of content and spitting out the results, so lets use a simple vector database called chroma db. but any vector db doesn't really want the full document, it wants the document chunked up into smaller pieces. So we need to split it up first. LangChain has something for this called a text splitter, so let's add it. 

'from langchain text splitter import recursivecharactertextsplitter' and now we can say how we want the text split up. 'text splitter = recursivecharactertextsplitter chunk size is 500 and overlap is 0'. so every chunk is going to be 500 characters and there is no overlap between the chunks. and then 'all splits = textsplitter.splitdocuments and feed it our data'

Now we can add those chunks to the database. But vector stores don't store just regular words, they store vectors. So the embeddings function is what converts the words. As of this recording, we don't have an embeddings function that langchain can use, so we will use the gpt4all embeddings. First install the python modules: 'pip install gpt4all and chromadb'

then in the file we add 'from langchain embeddings import gpt4all embeddings' and then import the database 'from langchain vectorstores import chroma'. Now instantiate that datastore. 'vectorstore = chroma.fromdocuments and documents = all splits and embedding is our gpt4allembeddings'.

In langchain, the central concept is the chain which connects a bunch of tasks together to complete a larger task. We can use the retrieval qa chain by adding to our file, 'from langchain.chains import retrievalqa' and then 'qa chain = retrievalqa from chain type and then ollama and retriever = vectorstore as retriever'. now we can ask our question to our document. 

'question is when was Hawaii's request for a major disaster declaration approved'. so 'qa chain with a query of the question'. and that’s it. pretty quickly we get an answer. Sometimes the results of the query isn't spot on. I think one of the biggest factors in this is how we split up the source information. I think its better to have a bit of overlap. So if we set our overlap to 50, we often get better answers, but you may need to play with that. 

and that’s how you can use ollama with langchain and python to ask a document a question. You could use a different loader to point to a directory, and you would probably want to keep the datastore up between questions so you don't have to keep importing. I'll share examples on how to do that later. And if you want to see how to do this with Javascript, keep an eye out for that video. Thanks so much for watching. goodbye.