Blog

Comparing Three AI Language Models: Can They Code?

Published on
March 21, 2024
Tessa McDaniel
Marketing Team Lead

Part two of the series comparing three of the biggest AI language models available. This time, how do they do with Python?

Welcome back! Last article, we saw how OpenAI’s ChatGPT, Google’s Gemini, and Microsoft’s Copilot handled my prompt, “Can you come up with some dinner recipes that are under 500 cal per serving?” (which was very handy for making dinner last week). But now I’m ready to try something a little less “lite” and get some help with coding. I’ll be asking them to write a Python function to change the color of the text shown in the terminals: "Can you write a Python function called color_print that prints text in the terminal in different colors or formatting using the colorama package and some defined ANSI escape sequences?". Quick disclaimer, I’m choosing Python because that’s the coding language I’m most familiar with and will be able to analyze more. If you want to use your coding skills in Virtuoso, JavaScript is your friend there. Alright, let’s go!

The Human

For reference, let’s look at my function. I tried to keep my function pretty simple, but notice that I am intentionally using the ANSI escape sequences. They may not be the most efficient (most of you will know that I don’t even need a function to use colorama, I can just run it as “print(Fore.RED + 'some red text')”, but I want to see if the GPTs will use ANSI as requested. Also, I’ve never been a Python expert, and I’m pretty rusty, so if there are areas that could’ve been more efficient… I really didn’t want to spend two hours figuring this out.

OpenAI ChatGPT

ChatGPT started off strong by reminding me to ensure that I have the colorama package installed (if you’ve never forgotten to install a package… I don’t believe you). Then it explained what colorama is, how it works with ANSI, and the different things you can do with colorama. I’ll get to the point: it did generate a successful function. It’s just not particularly efficient. I can’t even get the entire function in one screenshot, and for the second one, I skipped the rest of the dictionaries and went straight to the function code. You can see there’s a lot of extra text to the right, so it’s pretty lengthy. And it’s not following the PEP 8 style guide by starting with two blank lines, but I’m being particularly nitpicky. It also didn’t use ANSI escape sequences specifically as I asked.

The function itself is particularly bulky. It took the time to set the default formatting in the parameters instead of just leaving it as the default terminal output. Using the function is also clunky, as you have to type out your text_colour, bg_color, and style instead of just typing in the values, whereas it could’ve defined those in the initial function format.

Google Gemini

Gemini defined a few pieces of code and started with the function itself. It scores points for using the ANSI escape sequences as requested! PEP 8 would be disappointed in the lack of two blank spaces, but I suppose I didn’t specify a style guide. Then after the function, it gave me the instructions for use: import colorama (again), initialize it, and then print an example. Both the second import and specifying initializing colorama are unnecessary, but the function itself is pretty lean. When I’m using the function, I don’t have to specify all the variables, making it more user-friendly.

Microsoft Copilot

If you remember from the previous article, I used the specific cooking GPT for the prompt. Now, I’m just using generic Copilot. And unfortunately… it didn’t work. Let’s take it bit by bit. Firstly, it joins ChatGPT in that it doesn't use the ANSI escape sequences. And that problem becomes apparent when you realize that colorama doesn’t have a built-in “underline” style (hence the need for defining the ANSI escape sequences), which is why the example usage that requests “underline” as a style doesn’t work. The inclusion of the “underline” definition breaks the function entirely since it doesn’t exist in colorama. Having to specify the color and style when using the function is not very efficient, either. My last gripe is, once again, PEP 8 was not conformed to. 

Conclusion

Now, I’ve been pretty picky for someone whose Python skills are fairly lacking. But at the same time, it’s weirdly satisfying knowing that only one of the three GPTs produced a function that was more efficient than mine. Interestingly, that was also the only GPT that used the ANSI escape sequences (how many times have I said that phrase now?) I requested, and that was Gemini! It was pretty fun comparing all three GPTs, and I’ve learned that they’re definitely not one-size-fits-all, but they’re pretty useful in my personal life. I’ve even continued to get recipe recommendations from Copilot after writing part one, and I’ve made some tasty dinners! While I won’t be generating my Python functions (otherwise, how would I learn and get better?), comparing the functions all three created was quite insightful. Which GPT is your favorite?

Tags:

No items found.

Subscribe to our Newsletter