AI-Driven Software Development - Our Development Team's Experiment with ChatGPT
5 min read
July 21, 2023
In a recent coding-dojo session, our team of developers decided to push the limits of artificial intelligence and test the capabilities of ChatGPT. Our goal was to build a simple game entirely written by AI. In this case we were a group of eleven people, and we had two hours to explain the idea and see what we achieved.
What is a coding-dojo?
A coding-dojo is a collaborative learning environment where developers can work on coding challenges and improve their skills. Typically, groups will work on a specific problem or project, sharing knowledge and expertise to find the best possible solution. In our case, we developed a simple timer that randomly chooses a team member every five minutes to take ownership of managing the session, chat with the AI-driven coding environment, and develop the game, and also with a set of rules to maintain order and encourage teamwork and communication, for example, only the previous participant could give recommendations and help the active engineer.
The Experiment
We started by selecting ChatGPT, an AI model that uses machine learning to understand and respond to natural language. We used it to test the limits of AI-driven development by asking it to create game elements, mechanics, and fix problems and bugs.
Setup
The setup of the application was divided into three parts. A chat that consumes the ChatGPT API with some language development presets and some instructions, a timer with the participants’ list, and the game scene.
1. Coding Dojo Assistant with ChatGPT integrated
We use these instructions for the ChatGPT preset:
"You are a coding assistant that is tailored towards building games, you will always talk to software engineers. Don't explain the details. Provide only small code snippets. You will use the Phaser.io library and plain HTML and Javascript (ES6). You always answer in Markdown format."
This is what the chat looked like for us
2. Timer with participants list
The second part was a timer with the list of developers that, when the time ended, gave a warning and when it started, it randomly picked a new engineer.
3. Game scene
Finally, the last part was the game scene space, which was empty at the beginning, just the background image, but where later, the game elements were added.
We already had the assets needed at the beginning, and this is how the entire site looked at:
Developing with AI
We began by giving instructions to introduce the first elements: the water background, beaver and logs.
After that, we tried to put two logs coming from left to right, like in Flappy Bird. We had a lot of problems with size, movement, etc. for example, making the logs automatically appear at random time and at different heights was really hard for the AI, in fact, it did not succeed (we will talk about this in a bit).
This is the final stage!
The Results
Some more experienced engineers suggested good advice to guide the rest, like setting clear objectives, even in some cases, something controversial happened, which was that the active participant decided to delete all the previous code and start over! Although we did not manage to make a 100% functional game, a lot was learned about the manipulation of this type of tools, going to the classic triple question: “What, how, and how much?”
-
What?: What kinds of things should I ask ChatGPT? We attempted to submit complex requests to it, encompassing the development of entire applications or intricate functionalities, yet it consistently fell short in delivering satisfactory results. We will talk about this more in our conclusion.
-
How?: The way we phrased our prompts was very important. The choice of words, the contextual information provided, and even the preceding dialogue can greatly influence the outcome. Two people seeking the same information with different approaches will likely yield different results. This prompts the question: how specific do we need to be when creating a prompt?
-
How much?: The key here seemed to be striking a balance between brevity and specificity. For example, instead of using ambiguous instructions like "create functionality so that the logs appear on the screen as in Flappy Bird," it is more effective to provide clear and short indications of the log’s specific properties such as their movement patterns, starting and ending points, quantity, size, and other relevant attributes.
Conclusion
The creation of the application was in a very improvable state, but the experiment was very useful to know how to deal with a GPT AI. We were able to gain valuable insights into the capabilities and limitations of ChatGPT, in which cases it should be used and in which not, and above all, we should be careful with the credibility of the information. In many cases, it gave us an erroneous code that it corrected as we asked it to, so we believe that we must have previous knowledge to know if what it is telling us is useful or not.
We were also able to demonstrate the potential of AI-driven software development in a gaming scenario, and we are excited to continue exploring the possibilities of this technology. Stay tuned for more updates on our journey!