Using GenAI (Github Copilot) to build a GenAI service

Back on April 1st, I published my Faramir project, which is a (micro)service that provides a REST API that uses OpenAI to valid and format phone numbers. Faramir is written in Python and uses the Langchain library. I hadn’t actually written a Python-based REST API before, so naturally I turned to Github Copilot for help. This article documents some of my Github Copilot Chat interactions that helped me write the application.

Generate a python application that uses FastAPI

A quick Google search suggested that I use a framework called FastAPI. In Github Copilot chat, I used the following prompt ‘Generate a python application that uses FastAPI’. This is the response:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: str = None):
    return {"item_id": item_id, "q": q}

It included some helpful instructions:

… To run this application, save it as main.py and use the command uvicorn main:app –reload in your terminal. This will start a local server at http://127.0.0.1:8000.

Generate a pip3 requirements file

I then asked Copilot to generate a requirements.txt file for me.

fastapi==0.68.1
uvicorn==0.15.0

Which port is the application listening on?

I obviously wasn’t paying attention because I then asked Copilot which port the application was listening on.

The code snippet you provided does not specify the port the FastAPI application is listening on. However, by default, when you run a FastAPI application using Uvicorn, it listens on port 8000. You can specify a different port by using the –port option when starting Uvicorn, like so: uvicorn main:app –port 8080.

Handling a post request

The FastAPI example only handles GET requests. I asked Copilot to generate a handler for a POST request.

Here’s the code:

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Item(BaseModel):
    phoneNumber: str

@app.post("/items/")
def create_item(item: Item):
    return {"phoneNumber": item.phoneNumber}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: str = None):
    return {"item_id": item_id, "q": q}

One day I will read about pydantic and understand what it does. :-)

There’s more

Github Copilot also helped with other aspects of development by suggesting completions while I was writing the code. For example, it helped generate some Pytest tests for the service.

Summary: a useful yet computationally expensive to avoid RTFM

As you can see, Github Copilot was extremely helpful while I was writing my first Python-based service. But there are some drawbacks. I don’t have the knowledge to evaluate the suitability and safety of the libraries it recommends. Here’s an example of how that can go terrible wrong. Moreover, Github Copilot is a computationally very expensive alternative to RTFM. We really need a way to make libraries more usable that doesn’t involve ‘scanning all of humanity’s knowledge’ to generate each word.

To learn more

Please take a look at the Github repository.