Is this the future of AI assistants?

Introduction

What if you could automate any task simply by saying what you want to achieve?

You’ve probably already heard about ChatGPT - the system from OpenAI that can write and debug software, generate essays, write poetry and more. You can give it tasks in plain English (“please write a haiku on why AI is not a threat to humanity”). Some tasks are one-shot - you get back exactly what you need after one interaction. Most often in my experience though it take a few attempts. The generated code is not quite right, or the answer doesn’t quite make sense to you. Sometimes you need to give ChatGPT some more background knowledge because it can’t search the internet by itself.

What if you could automate all that?

What if you could give ChatGPT a goal and it would work out what it needed to do to achieve that goal and then do it. Without you needing to do anything else?

That’s the promise of Auto-GPT systems.

What is Auto-GPT?

I’ll talk more about how it works later, but the core idea is actually pretty simple: You give the system a role and some goals and it attempts to work out how to achieve those goals for you. Now you might wonder “How is this different from ChatGPT?”. It’s true that you can tell ChatGPT what you want and it will attempt to do it for you. However, it’s limited to the knowledge it was trained with and all it can do is provide you with a text output in a web browser. If it’s not quite right or it needs more input you need to provide that. Auto GPT provides a way for GPT4 to search the Internet, pull information it needs from web pages and write files to your local disk. What’s more it attempts to work out when it’s achieved a goal and continues until that’s done (or you run out of either patience or money).

Auto-GPT create a writing career for me

There are some great demos showing what can be acheived such as market research for waterproof shoes but I’ll show some of my own experience.

The first thing you need to go is give the AI agent a name and a role. You can then give it up to five goals. Here’s what I tried:

Name: WritingCoach

Role: You are an expert writing teacher with a long history of helping aspiring writers achieve their goals and reach financial independence through writing articles on medium.com

Goals:

  • Create a plan for a software engineer to become a successful writer about AI
  • The plan must not require that the software engineer devotes more than on hour per day to writing
  • The plan must allow the software engineer to transition from a full-time job as a software engineer to a full-time job as a writer
  • The plan must allow the software engineer to make at least $100,000 per year from writing
  • It must be possible to complete the plan in five years or less

Hey, I figured why not be ambitious!

This is what the system responded:

1
2
3
4
5
6
7
8
THOUGHTS: I think the first step is to research the current market for AI writing and identify the most lucrative niches. This will help us create a plan that is tailored to the software engineer's strengths and interests. I can use a Google search to find this information.
REASONING: Researching the market will help us identify the most profitable niches and tailor our plan to the software engineer's strengths and interests.
PLAN: 
-  Use a Google search to research the current market for AI writing
-  Identify the most lucrative niches
-  Tailor the plan to the software engineer's strengths and interests
CRITICISM: I need to ensure that I am thorough in my research and consider all possible niches. I also need to keep in mind that the software engineer may have limited writing experience, so I need to create a plan that is achievable for them.
SPEAK: I think we should start by researching the current market for AI writing to identify the most lucrative niches. This will help us create a plan that is tailored to the software engineer's strengths and interests.

That seems fairly sensible. Notice how the response is organised into several sections. This is how Auto GPT is able to process then response from GPT.

Next it asks my approval to execute a command. You can run Auto-GPT so that it doesn’t prompt for approval for every command, but I’m not that trusting!

1
2
3
NEXT ACTION:  COMMAND = google  ARGUMENTS = {'input': 'most lucrative niches for AI writing'}
-=-=-=-=-=-=-= COMMAND AUTHORISED BY USER -=-=-=-=-=-=-= 
SYSTEM:  Command google returned: b'[\n    {\n        "title": "35 High Paying Freelance Writing Niches for 2023 (Yearly Salaries)",\n        "href": "https://elnacain.com/blog/freelance-writing-niches/",\n        "body": "Your writing niche is a focused area in writing like writing press releases or blog posts where you are experienced in and knowledgeable of writing about. You can have a broad freelance writing niche like finance and niche down to specific topics within finance like cryptocurrency or investments."\n    },\n    {\n        "title": "17 Freelance Writing Niches That Still Pay Big Bucks in 2023",\n        "href": "https://smartblogger.com/freelance-writing-niches/",\n        "body": "Freelance Writing Niches: 9 High-Paying Content Types Remember, the more your writing can impact a client\'s sales, the higher it pays. So all of these content types are either under the umbrella of content marketing or copywriting. 1. Long-form Blog Posts or Ultimate Guides"\n    },\n    {\n        "title": "12 Most Profitable Freelance Writing Niches (In 2023) - Rafal Reyzer",\n        "href": "https://rafalreyzer.com/most-profitable-freelance-writing-niches/",\n        "body": "12 Most Profitable Freelance Writing Niches 1. Finance Almost 70% of adults use banking services worldwide. Finance is one of the main pillars of our civilization, and that\'s exactly why there\'s rarely ever been a shortage of work in this field, even during financial crises. This vast field encompasses a lot of topics for writers to focus on."\n    },\n    {\n        "title": "Top 12 Most Profitable Freelance Writing Niches",\n        "href": "https://writingcooperative.com/top-12-most-profitable-freelance-writing-niches-a3b0eb504eba",\n        "body": "This particular niche is one of the most profitable freelance writing niches. It encompasses anything from: Computers Mobile devices Blockchain Cryptocurrency And so forth. The tech industry is a moving train that no one can stop; and so is the need for freelance writers."\n    },\n    {\n        "title": "10 Most Lucrative Freelance Writing Niches for 2023 - Frayed Passport",\n        "href": "https://frayedpassport.com/10-most-lucrative-freelance-writing-niches-for-2023-and-beyond/",\n        "body": "Here are the most lucrative freelance writing niches by content. 1. White Papers An experienced white paper writer can charge $5,000 to $7,000 per whitepaper, according to industry experts. That\'s because, in the digital world, white papers can work for years to help businesses generate high-quality leads and achieve sales."\n    },\n    {\n        "title": "21 Most Profitable Freelance Writing Niches (In 2023)",\n        "href": "https://outandbeyond.com/most-profitable-freelance-writing-niches/",\n        "body": "The finance niche is undoubtedly one of the most profitable freelance writing niches for the simplest reason, money is and will always be relevant. There are various components in the finance niche for you to tackle such as: Personal Finance Budgeting Banks and Credit Card Mortgage Financial stocks Accounting"\n    },\n    {\n        "title": "The 13 Most Profitable Writing Niches of 2021 - LinkedIn",\n        "href": "https://www.linkedin.com/pulse/13-most-profitable-writing-niches-2021-colleen-welsch",\n        "body": "If you\'re looking for a profitable writing niche, you\'ll have luck with one of these: 1. Finance Few people understand finance, but everybody has questions about it. Thus, there is a great..."\n    },\n    {\n        "title": "15 Best Niche Markets To Make REAL Money Online - Niche Pursuits",\n        "href": "https://www.nichepursuits.com/best-niche-markets/",\n        "body": "In terms of niche marketing this is fantastic. It means there\'s an existing market for language tuition software, online language coaching services, and everything else related to learning a new language. Market value: US$43 billion per year. 2."\n    }\n]'

Based on the results of its google search Auto-GPT comes up with the next stage of its plan:

1
2
3
4
5
6
7
8
THOUGHTS: Now that we have identified some of the most lucrative niches for AI writing, we need to determine which ones align with the software engineer's interests and expertise. We can use a GPT agent to help us with this task.
REASONING: Aligning the niches with the software engineer's interests and expertise will increase their motivation and likelihood of success. A GPT agent can help us quickly narrow down the options.
PLAN: 
-  Use a GPT agent to determine which niches align with the software engineer's interests and expertise
-  Narrow down the options to 2-3 niches
-  Research these niches further to determine their profitability and demand
CRITICISM: I need to ensure that the GPT agent is properly trained and that we consider all possible niches. I also need to keep in mind that the software engineer may have limited writing experience, so I need to create a plan that is achievable for them.
SPEAK: Now we need to determine which niches align with the software engineer's interests and expertise. We can use a GPT agent to help us with this task.

Now Auto-GPT wants to spawn a new agent. Agents are a way for Auto-GPT to pursue several threads of investigation or action at once. Each agent has its own context and so does not have to remember the global state and won’t be confused by information unrelated to its task.

1
2
3
NEXT ACTION:  COMMAND = start_agent  ARGUMENTS = {'name': 'AI Writing Niches', 'task': "Determine which AI writing niches align with the software engineer's interests and expertise", 'prompt': "Which AI writing niches align with the software engineer's interests and expertise?"}
-=-=-=-=-=-=-= COMMAND AUTHORISED BY USER -=-=-=-=-=-=-= 
SYSTEM:  Command start_agent returned: Agent AI Writing Niches created with key 0. First response: As an AI language model, I cannot determine the specific interests and expertise of a software engineer. However, some AI writing niches that may align with the interests and expertise of software engineers include technical writing, software documentation, and coding tutorials.

This triggers another round of thinking:

1
2
3
4
5
6
7
8
THOUGHTS: The GPT agent has provided us with some initial niches that may align with the software engineer's interests and expertise. We can now use a Google search to research these niches further and determine their profitability and demand.
REASONING: Researching the niches further will help us determine which ones are most profitable and in demand. A Google search is a quick and efficient way to gather this information.
PLAN: 
-  Use a Google search to research the niches provided by the GPT agent
-  Determine which niches are most profitable and in demand
-  Narrow down the options to 2-3 niches
CRITICISM: I need to ensure that I am thorough in my research and consider all possible niches. I also need to keep in mind that the software engineer may have limited writing experience, so I need to create a plan that is achievable for them.
SPEAK: Now we need to research the niches provided by the GPT agent to determine their profitability and demand. We can use a Google search to gather this information.

This goes on for a couple more rounds until it decides on the following topic areas:

  1. Content creation for e-commerce websites
  2. Social media content creation and management
  3. SEO content creation and optimization
  4. Email marketing campaigns and newsletters
  5. Product descriptions and reviews
  6. Technical writing for software and technology companies
  7. Academic writing and research papers
  8. Creative writing for fiction and non-fiction books
  9. Copywriting for advertising and marketing agencies
  10. Legal writing and contract drafting

Unfortunately, these don’t seem to have a lot to do with writing about AI. It’s clear by this point that it’s unlikely to do what I want, but I was curious how far it will go so I let it continue for a bit.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
THOUGHTS: Based on the Google search results, it seems that technical writing for software and technology companies is a promising niche for the software engineer to write about. We should now create a plan for the software engineer to establish themselves as an expert in this niche and start earning income from their writing.
REASONING: Technical writing for software and technology companies is a niche that is in high demand and can be very profitable. By establishing themselves as an expert in this niche, the software engineer can attract clients and start earning income from their writing.
PLAN: 
-  Create a portfolio of technical writing samples
-  Build a website to showcase their writing and expertise
-  Network with software and technology companies to find potential clients
-  Write articles and blog posts on technical writing topics to establish themselves as an expert in the field
-  Use social media to promote their writing and expertise
CRITICISM: I need to ensure that the plan is achievable for the software engineer and that they have the necessary skills to execute it. I also need to consider the time constraints and ensure that the plan does not require more than one hour per day of writing.
SPEAK: Based on the Google search results, technical writing for software and technology companies seems like a promising niche for the software engineer. We should create a plan for them to establish themselves as an expert in this niche and start earning income from their writing.

THOUGHTS: Let's start by creating a portfolio of technical writing samples. We can use GitHub to create a repository for the software engineer's writing samples.
REASONING: Creating a portfolio of technical writing samples is an important step in establishing oneself as an expert in the field. GitHub is a great platform for hosting writing samples and showcasing the software engineer's technical writing skills.
PLAN: 
-  Create a GitHub account
-  Create a repository for the software engineer's technical writing samples
-  Upload at least 5 technical writing samples to the repository
-  Organize the repository to make it easy for potential clients to find relevant samples
CRITICISM: I need to ensure that the writing samples are of high quality and showcase the software engineer's technical writing skills. I also need to ensure that the repository is well-organized and easy for potential clients to navigate.
SPEAK: Let's start by creating a portfolio of technical writing samples. We can use GitHub to create a repository for the software engineer's writing samples.

So that’s quite neat. It decided that a writer needed a portfolio and it was going to go ahead and generate one for me. It also decided that it was going to use GitHub to host the portfolio and attempted to create a GitHub account for me (which failed BTW).

Note that in my original goals I said nothing about GitHub or wanting a portfolio.

After watching it flail around trying to create a GitHub account for a while I stopped it.

At the time I tried the above I didn’t have access to GPT4 and so had to use GPT3.5. When I got access to GPT4 I tried again. It didn’t succeed this time either, but it came up with a better list of topics:

  1. Machine Learning
  2. Natural Language Processing
  3. Computer Vision
  4. Robotics
  5. Expert Systems
  6. Speech Recognition
  7. Neural Networks
  8. Deep Learning
  9. Reinforcement Learning
  10. AI Ethics

So, are we all obsolete yet?

I tried another couple of scenarios with varying levels of success. In one instance I asked Auto-GPT to create a cross-compilation toolchain for me using crosstool-ng. It got as far as cloning the crosstool-ng repo and attempting to configure it.

I also asked it to create a mobile app for me using Flutter. In one attempt it generated python code to run a shell process to create a flutter project, but then failed to execute it correctly.

1
2
3
import subprocess

subprocess.run(['flutter', 'create', 'gym_book'])

It then seemed to get stuck in a loop attempting to execute the file, failing, but then trying again.

I restarted Auto-GPT and this time it attempted to execute the flutter command as a python file which also (unsurprisingly) failed. Once again it wasn’t able to recover from this.

I decided to try one more time, but this time attempt to guide Auto-GPT to get past the step it was failing at before. While doing this I realised that my original prompt stated that flutter was already installed (which it was). But Auto-GPT runs python code in a docker image and so couldn’t access the local installation. This time Auto-GPT browsed the flutter website to find out how to install flutter but then crashed as it exceeded the GPT context length.

Auto-GPT is currently hard-coded to run the python:3-alpine image. I found a docker image with the flutter SDK, but it didn’t contain python so at that point I decided to call it a day.

It’s possible I’m terrible at crafting goals for Auto-GPT. It does get tantalisingly close to doing useful stuff and I plan to continue experimenting with it, but I think my job is safe for this year.

How does it work?

We want to get from a single command and response to a chain of thoughts and actions that be used to achieve a goal. Chain of thought prompting has been shown to get better results in some cases by asking the LLM to show intermediate steps to get to an answer. ReAct extends this byinstructing the LLM to interleave reasoning and task specific actions and teaching it how to request that those actions are performed. If the LLM chooses an action, the software takes that action and feeds the results back to the LLM.

Auto-GPT uses a prompt that instructs GPT to reply using a fixed format and that it may use one of a number of built-in commands.

Response contents

1
2
3
4
5
6
7
8
9
10
self.response_format = {
    "thoughts": {
        "text": "thought",
        "reasoning": "reasoning",
        "plan": "- short bulleted\n- list that conveys\n- long-term plan",
        "criticism": "constructive self-criticism",
        "speak": "thoughts summary to say to user",
    },
    "command": {"name": "command name", "args": {"arg name": "value"}},
}

Code to generate the prompt with the JSON formatted response:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
def generate_prompt_string(self) -> str:
    """
    Generate a prompt string based on the constraints, commands, resources,
        and performance evaluations.

    Returns:
        str: The generated prompt string.
    """
    formatted_response_format = json.dumps(self.response_format, indent=4)
    return (
        f"Constraints:\n{self._generate_numbered_list(self.constraints)}\n\n"
        "Commands:\n"
        f"{self._generate_numbered_list(self.commands, item_type='command')}\n\n"
        f"Resources:\n{self._generate_numbered_list(self.resources)}\n\n"
        "Performance Evaluation:\n"
        f"{self._generate_numbered_list(self.performance_evaluation)}\n\n"
        "You should only respond in JSON format as described below \nResponse"
        f" Format: \n{formatted_response_format} \nEnsure the response can be"
        " parsed by Python json.loads"
    )

These commands include:

  • searching using google
  • creating agents
  • reading from and writing to files
  • opening a web browser and extracting the contents of a web page
  • sending a tweet via twitter
  • generating an image using Dall-e
  • cloning a GitHub repo
  • running python code (in a docker container)

Auto-GPT works like this:

  • sends the current state to ChatGPT API
  • receives the response in a JSON format it knows how to parse
  • executes the command (if any) in the response
  • feeds back the result of the command, the next time around the loop

It keeps looping like this until it decides the goals have been achieved, at which point it stops.

There is more to it than this. For example Auto-GPT can integrate with vector databases like Pinecone and use that as a memory.

There can never be only one

Although Auto-GPT was the first project I came across there are others, including

Jarvis is particularly interesting as it can make use of other AI models from the Hugging Face library. Jarvis selects models to use for each task, executes the task using a model and then feeds the results back into the next round. In other words it expands on Auto-GPT by using other AI models, not just simple commands, to achieve a goal.

There is also LangChain a framework for developing applications powered by language models. LangChain makes it easy to implement ReAct logic.

Concerns

As with any piece of software there are risks. Here’s a few of the more obvious ones.

You are giving something control over your local computer - this is true of all software to some degree. Instead of software that performs a known task you have a large language model that could decide to run something damaging. The fixed set of built-in commands means that it shouldn’t be possible, for Auto-GPT to delete all your files, for example. However, anything that can write to your local disk could potentially scribble all over something you care about.

It could rack up large bills - the OpenAI API is not free and Auto-GPT does appear to sometimes get stuck in loops. There is the potential for it to rack up large bills if you let it make large numbers of API calls. You can protect yourself by configuring a budget on the OpenAI platform and/or when you create a new Auto-GPT agent.

Conclusions

Has Auto-GPT made developers obsolete? Not yet, but it is an interesting and logical development of GPT technology. I have not yet achieved an AI powered life of leisure.

Poking at Auto-GPT also made me aware of vector databases and so has already helped to expand my knowledge.

I also intend to spend more time investigating what can be done with frameworks like LangChain.

There is a lot of interesting work happening in this space. Be sceptical of the hype, but keep open-minded.