I asked ChatGPT to do my job

I’m a programmer and always look for a way to improve my productivity to find more time to do something more meaningful with my life like arguing with strangers on Twitter. To do that I use a lot of tools, including IDE, plugins, and recently GitHub Copilot. The AI helps me complete my code more than your wife can complete your sentences but It’s not perfect. It still needs a programmer to think about how to code it from the requirements and many times the code generated from the GitHub Copilot can’t even run. I mostly use it for auto-completing my well-thought code.

ChatGPT finally come. A chatbot that is capable to talk and engage in deep conversation and remember what you said, what it said, can be correct. I tried it on the long weekend, mostly trying to break it and explore the limit of AI and know it is capable of generating functional code.

During my work hour, I got the idea to use ChatGPT to completely do my job. Basically, turn a requirement into a real-world code. My requirement is super simple: A RESTAPI that has the ability to globally pause and resume calling others RESTAPI (AKA circuit breaker). I decide to provide the requirement as a conversational instead of an essay as a real product owner does.

First I asked the ChatGPT to make a simple dummy API that does something to be a normal functional RESTAPI.

Me: Write a Python FastAPI to return the user's remote IP address.

ChatGPT:

Here is an example of a FastAPI app that returns the user’s remote IP address:

from fastapi import FastAPI
from fastapi.middleware.trustedhost import TrustedHostMiddleware
from starlette.middleware.httpsredirect import HTTPSRedirectMiddleware

app = FastAPI()
app.add_middleware(TrustedHostMiddleware, allowed_hosts=["example.com"])
app.add_middleware(HTTPSRedirectMiddleware)

@app.get("/")
async def get_ip_address(request):
    return {"remote_ip": request.client.host}

This app uses the TrustedHostMiddleware to ensure that the request is coming from a trusted host (i.e. "example.com" in this case). It also uses the HTTPSRedirectMiddleware to automatically redirect HTTP requests to HTTPS.

To access the user’s IP address, the app uses the request.client.host attribute. This returns the IP address of the user's client, which is the remote IP address in this case.

At this point, I was already impressed with the result. It describes what the code does really well but I won’t bother running it since it is not the real requirement. I continue to ask for my main requirement.

ME: Add a feature to pause and resume the API

ChatGPT:

Here is an example of how you could add a feature to pause and resume a FastAPI app:

from fastapi import FastAPI
from fastapi.middleware.trustedhost import TrustedHostMiddleware
from starlette.middleware.httpsredirect import HTTPSRedirectMiddleware
from starlette.requests import Request
from starlette.responses import Response

app = FastAPI()
app.add_middleware(TrustedHostMiddleware, allowed_hosts=["example.com"])
app.add_middleware(HTTPSRedirectMiddleware)

PAUSED = False

@app.middleware("http")
async def pause_middleware(request: Request, call_next):
    global PAUSED
    if PAUSED:
        return Response("The API is currently paused", status_code=503)
    response = await call_next(request)
    return response

@app.get("/pause")
async def pause():
    global PAUSED
    PAUSED = True
    return {"message": "The API has been paused"}

@app.get("/resume")
async def resume():
    global PAUSED
    PAUSED = False
    return {"message": "The API has been resumed"}

@app.get("/")
async def get_ip_address(request):
    return {"remote_ip": request.client.host}

This app adds a PAUSED global variable that is initially set to False. It also adds a pause_middleware function that checks the value of the PAUSED variable and returns a response with a status code of 503 (Service Unavailable) if the API is paused. This middleware is added to the app using the app.middleware decorator, which allows it to run for every HTTP request.

The app also adds two new endpoints: /pause and /resume, which can be used to pause and resume the API respectively. These endpoints simply toggle the value of the PAUSED variable.

When the API is paused, any requests to it will receive a response with a status code of 503 and a message indicating that the API is paused. When it is resumed, requests will be handled as usual.

It keeps impressing me. It added a middleware to intercept all requests and check the pausing status with a global variable PAUSED and return HTTP Error 503 when it is paused. I actually copy the code and run it inside my real work but I don't like the error result which is just plain text. I could change it myself easily but I could just ask ChatGPT to change it.

ME: Change the middleware Response to JSON

ChatGPT:

Here is an example of how you could modify the middleware in the previous example to return a JSON response instead of a plain text response:Copy code

from fastapi import FastAPI
from fastapi.middleware.trustedhost import TrustedHostMiddleware
from starlette.middleware.httpsredirect import HTTPSRedirectMiddleware
from starlette.requests import Request
from starlette.responses import JSONResponse

app = FastAPI()
app.add_middleware(TrustedHostMiddleware, allowed_hosts=["example.com"])
app.add_middleware(HTTPSRedirectMiddleware)

PAUSED = False

@app.middleware("http")
async def pause_middleware(request: Request, call_next):
    global PAUSED
    if PAUSED:
        return JSONResponse(
            {"message": "The API is currently paused"},
            status_code=503
        )
    response = await call_next(request)
    return response

@app.get("/pause")
async def pause():
    global PAUSED
    PAUSED = True
    return {"message": "The API has been paused"}

@app.get("/resume")
async def resume():
    global PAUSED
    PAUSED = False
    return {"message": "The API has been resumed"}

@app.get("/")
async def get_ip_address(request):
    return {"remote_ip": request.client.host}

This app is similar to the previous example, but instead of returning a Response object in the middleware, it returns a JSONResponse object. This allows you to specify the JSON body of the response and the HTTP status code.

It can remember the code it generated. I can correct the code by just talking to it but the code had a major flaw. It cannot resume the APIs after the first pause called when called /resume the middleware always response 503 and will not be able to change the PAUSED flag back. I asked vaguely about the problem.

ME: But how it can resume if the middleware always responds error 503?

ChatGPT:

The middleware only returns a response with a status code of 503 if the PAUSED variable is True. If the PAUSED variable is False, the middleware simply passes the request on to the next piece of middleware or the endpoint handler, and the response is handled as usual.

This means that when the /resume endpoint is called and the PAUSED variable is set to False, the middleware will not return a response and the request will be handled by the endpoint handler. This allows the API to be resumed and start serving requests again.

Here is an example of what this might look like:

The PAUSED variable is initially set to False

A request is made to the / endpoint

The pause_middleware function is called and checks the value of the PAUSED variable

The PAUSED variable is False, so the middleware passes the request on to the get_ip_address endpoint handler

The get_ip_address endpoint handler returns a response with the user's IP address

The response is returned to the client

Now suppose the /pause endpoint is called, which sets the PAUSED variable to True. If a request is made to the / endpoint, the following will happen:

The PAUSED variable is set to True

A request is made to the / endpoint

The pause_middleware function is called and checks the value of the PAUSED variable

The PAUSED variable is True, so the middleware returns a JSON response with a status code of 503 and a message indicating that the API is paused

The response is returned to the client

If the /resume endpoint is called, which sets the PAUSED variable to False, subsequent requests to the / endpoint will be handled as usual, because the middleware will pass them on to the endpoint handler instead of returning a response.

The response basically describes a simple boolean logic like I was 10 and fixing nothing. I have to ask it again but more specifically about the logic flaw.

ME: But it cannot use /resume to resume the API while PAUSED variable is True because the middleware will always response 503 error.

ChatGPT:

You are correct that if the PAUSED variable is True, the middleware will always return a response with a status code of 503 and the /resume endpoint will not be able to resume the API.

However, you can modify the middleware to only return a response with a status code of 503 if the request path is not /resume. This will allow the /resume endpoint to be called, even when the PAUSED variable is True.

Here is an example of how you could do this:

from fastapi import FastAPI
from fastapi.middleware.trustedhost import TrustedHostMiddleware
from starlette.middleware.httpsredirect import HTTPSRedirectMiddleware
from starlette.requests import Request
from starlette.responses import JSONResponse

app = FastAPI()
app.add_middleware(TrustedHostMiddleware, allowed_hosts=["example.com"])
app.add_middleware(HTTPSRedirectMiddleware)

PAUSED = False

@app.middleware("http")
async def pause_middleware(request: Request, call_next):
    global PAUSED
    if PAUSED and request.url.path != "/resume":
        return JSONResponse(
            {"message": "The API is currently paused"},
            status_code=503
        )
    response = await call_next(request)
    return response

@app.get("/pause")
async def pause():
    global PAUSED
    PAUSED = True
    return {"message": "The API has been paused"}

@app.get("/resume")
async def resume():
    global PAUSED
    PAUSED = False
    return {"message": "The API has been resumed"}

@app.get("/")
async def get_ip_address(request):
    return {"remote_ip": request.client.host}

In this app, the pause_middleware function checks the value of the PAUSED variable, but only returns a response with a status code of 503 if the PAUSED variable is True and the request path is not /resume. This allows the /resume endpoint to be called even when the API is paused.

When the /resume endpoint is called, it sets the PAUSED variable to False, which causes the middleware to stop returning a response with a status code of 503 for subsequent requests. This allows the API to be resumed and start serving requests again.

This shows how it can be confidently wrong and shamelessly admit its mistake. Also, can correct itself if given a guide. At the same time, I try to correct the code myself using GitHub Copilot. It completes the middleware condition with request.url.path != “/pause” which is wrong. I copy the code from ChatGPT and added it to a real-world project.

Now I am pretty sure that ChatGPT can code from requirements and can be adapted to change. But I feel like AI coding is an interim solution. Software development in the far future might not need a programming language. We might just talk in natural language to an AI and it generated a working executable binary that does what we said.