Fork 1
13 GiB materialized 12 GiB stored


A langchain demo in xethub

A langchain project using openai to answer questions about fairy tales by indexing text files, querying a sqlite database, and running python code.


$ python -m venv .venv
$ . .venv/bin/activate
$ pip install -r requirements.txt

If you want to try searching the internet:
Sign here

pip install google-search-results

Checkout index and sql database

git xet checkout -- index
git xet checkout -- data


Search docs

To search within a group of files, we must first split them to small chunks, embed them as vectors, and save to an index.
On query time, we find the most relevant chunks, retrieve them and construct a prompt with the query and the context.


With langchain, building a doc search engine is as simple as:

from langchain.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator
import glob
import itertools

loaders = list(itertools.chain(*[TextLoader(file_path) for file_path in glob.glob(f'data/*.txt')]))
index = VectorstoreIndexCreator().from_loaders(loaders)
index.query_with_sources("Who was Pinocchio's father?")

In this repo we made an index helper which wrap it for training and querying workflows.

from src.index import Index

index = Index(index_path='index').fit("data", reset=True)
index = Index.load('model_path')
index.query("Who was Pinocchio's father?")
  • python src/train.py would do the same
  • If you clone the repo as is, the index is already populated.
  • Chunking-strategies

SQL queries

We saved a sqlite db with imdb dataset in the same data folder. With langchain we can query it with natural language.

from langchain import OpenAI, SQLDatabase
from langchain.chains import SQLDatabaseSequentialChain

db_chain = SQLDatabaseSequentialChain.from_llm(llm=OpenAI(temperature=0),

db_chain.run("How many movies are there?")


We can also run python code with langchain using PythonREPL (Read-Eval-Print Loop).

from langchain.agents import Tool
from langchain.utilities import PythonREPL

repl_tool = Tool(
    description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.",
repl_tool.run("print('Hello World')")

Search the internet

  • You'll need a serp api key
  • pip install google-search-results
from langchain.utilities import SerpAPIWrapper

search = SerpAPIWrapper()
print(search.run("Obama's first name?"))

If you don't want, we can use wikipedia instead.

pip install wikipedia

from langchain.utilities import WikipediaAPIWrapper

print(WikipediaAPIWrapper().run("Who was Pinocchio's father?"))


A Chatbot is created by holding on to the last few messages plus a system basic message.

A quick example:

from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

llm = OpenAI(temperature=0)
conversation = ConversationChain(

conversation.predict(input="how are you doing?")

A more ChatGPT-like can be by adjusting the system prompt

from langchain import OpenAI, LLMChain, PromptTemplate
from langchain.memory import ConversationBufferWindowMemory
import pathlib

prompt = PromptTemplate(
    input_variables=["history", "human_input"],
    template=pathlib.Path("prompts/markdown_assistant.txt").read_text()  # I made her a bit sassier

chatgpt_chain = LLMChain(
print(chatgpt_chain.predict(human_input="Do you believe in the moon landing?"))
print(chatgpt_chain.predict(human_input="What is in area 51?"))

Combining everything

We can use all of these capabilities as tool and provide them to our agent.

from langchain import OpenAI, SerpAPIWrapper
from langchain.agents import Tool, initialize_agent
from langchain.agents import AgentType
from langchain.tools.python.tool import PythonREPLTool

llm = OpenAI(temperature=0)

tools = [
        description="useful for when you need to answer questions about current events. You should ask targeted questions"
        description="useful for when you need to calculate somthing using programing"
mrkl = initialize_agent(tools, llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
mrkl.run("What is the capital of France? and use python to get a hash of it")

Run the app

gradio app.py

File List Total items: 14
Name Last Commit Size Last Modified
data working demo 4 months ago
docs update talk with react paradigm 3 months ago
index final training 4 months ago
logging/sqlitecache final training 4 months ago
notebooks working demo 4 months ago
prompts working demo 4 months ago
src final training 4 months ago
tests working demo 4 months ago
.gitattributes Initial commit 79 B 4 months ago
.gitignore working demo 3.9 KiB 4 months ago
app.py final training 1.7 KiB 4 months ago
config.py final training 444 B 4 months ago
readme.md fix title address 5.7 KiB 4 months ago
requirements.txt working demo 1.9 KiB 4 months ago

Repository Size

Materialized: 13 GiB
Stored: 12 GiB

Activity 14 commits

File Types