A comparison between Chat GPT - Swift Copliot GPT, Cursor (using Claude 3.5 Sonnet) and Microsoft Copilot in their ability to code a explicitly defined SwiftUI feature using the strongly opinionated architecture pattern of The Composable Architecture.
Model | Prompts | Compiled Immediately | Build Errors | Changes Required Before Compiling | Score Criteria |
---|---|---|---|---|---|
🏆 Cursor Claude 3.5 Sonnet (2nd attempt) |
1 | Yes | 0 | 0 | 77.78% |
Chat GPT-4 Swift Copilot GPT | 5 | Yes | 0 | 0 | 55.56% |
Chat GPT-4 Swift Copilot GPT | 1 | No | 5 | 1 file changed, 5 insertions(+), 6 deletions(-) | |
Cursor Claude 3.5 Sonnet (1st attempt) |
7 | No | 6 | 1 file changed, >20 lines modified | |
Microsoft Copilot (GPT-4) | 2 | No | 14 | Unable to Build |
The prompts have remained fairly similar throughout the tests, I have made some variations due to the context in which they are given. Eg I have not used “As a Principle iOS Engineer” when prompting within a development IDE. Also, the initial prompt to Chat GPT was less strict in the requirements, however experience showed that better results could be obtained by being more detailed and specific in the language in the prompts
💰 Pricing (ATOW): Free, Pro - $20 p/m, Business - $40 p/m
Cursor looks and feels like coding in VSCode, which is unsurprising as it’s a fork of VSCode, there is even an official migration guide. What is really useful about Cursor is the way you can ask for code like any other LLM, but it then works out how to automatically integrate the results with your existing codebase.
For iOS development, we still generally want to use XCode to build and run the code, so the workflow I have been using is to have both Cursor and XCode open, writing in Cursor, saving and then ‘Cmd-Tab’ to swap to XCode to Build and Run.
I need you to write Swift code for a SwiftUI feature using The Composable Architecture (TCA) pattern - https://github.com/pointfreeco/swift-composable-architecture that includes:
* A TextField where the user can input a topic.
* A Button that, when tapped, triggers an action to load questions related to the entered topic.
* A List that displays all the questions fetched from a mock API based on the topic.
* A Loading Spinner (ActivityIndicator) that appears while the questions are being fetched.
* If no questions are found, the list should appear empty.
Use The Composable Architecture (TCA) structure, ensuring that the code is:
* Modular, clean, and testable.
* Built with the following components:
* View: A SwiftUI view that displays the text field, button, list of questions, and loading spinner.
* Feature: A Struct that contains the reducer logic, handles how actions modify the state, including setting loading state, handling results, and updating the view accordingly.
* Within the Feature struct
* State: Holds the user's topic input, the loading state, the fetched questions, and a flag for handling empty results.
* Action: Triggers for text input changes, button taps, starting the API call, receiving the API response, and finishing loading.
* APIClient: Simulates a simple API call using a delay. This should be a dependency that can be used by the Feature
Ensure that:
* If the API call returns no questions, display an empty list with no error message.
* While the API call is in progress, show a loading spinner in place of the list.
* You provide comments to explain each part of the architecture.
* The code is concise, avoiding unnecessary complexity.
* Do not include unit tests in the code. Return the complete code as a single file that can be copied directly and run in an existing XCode project
Did not compile ❌, required 7 subsequent prompts to compile
same as 1st attempt - see above
Compiled immediately ✅, highest scoring result 🏆
See Cursor Generated Code Here
💰 Pricing (ATOW): Free, Plus - $20 p/m
As a Principle iOS Engineer I want you to help me write functional Swift code that compiles.
Architecture Pattern: The Composable Architecture - https://github.com/pointfreeco/swift-composable-architecture
Requirements:
1. Create a Swift UI View that has one text field to enter a topic, and one button to load questions using the text entered by the user.
Assumptions: Assume I already have an XCode project created and that I have already imported The Composable Architecture dependency in my project
Output: Firstly Provide the Names and functions of the entities that will be needed to build the feature
Make the Reducer functionality be part of a struct or class?
modify the code above to move the QuestionState QuestionAction and QuestionEnvironment inside the QuestionFeature
Actually please take the Environment out of the QuestionFeature, and make it a separate struct called QuestionAPI
Now please provide the code in a single file containing the QuestionView, the QuestionFeature and the QuestionAPI
Compiled immediately ✅
See Chat GPT Generated Code Here
Stage 1 - Crafted The Prompt (used 1st stage to ask Chat GPT to recursively critique the prompt to build the prompt below)
You will act as an expert Swift developer specializing in The Composable Architecture (TCA) pattern - https://github.com/pointfreeco/swift-composable-architecture
I need you to write Swift code for a SwiftUI feature using TCA that includes:
* A TextField where the user can input a topic.
* A Button that, when tapped, triggers an action to load questions related to the entered topic.
* A List that displays all the questions fetched from a mock API based on the topic.
* A Loading Spinner (ActivityIndicator) that appears while the questions are being fetched.
* If no questions are found, the list should appear empty.
Use The Composable Architecture (TCA) structure, ensuring that the code is:
* Modular, clean, and testable.
* Built with the following components:
* View: A SwiftUI view that displays the text field, button, list of questions, and loading spinner.
* Feature: A Struct that contains the reducer logic, handles how actions modify the state, including setting loading state, handling results, and updating the view accordingly.
* Within the Feature struct
* State: Holds the user's topic input, the loading state, the fetched questions, and a flag for handling empty results.
* Action: Triggers for text input changes, button taps, starting the API call, receiving the API response, and finishing loading.
* APIClient: Simulates a simple API call using a delay. This should be a dependency that can be used by the Feature
Ensure that:
* If the API call returns no questions, display an empty list with no error message.
* While the API call is in progress, show a loading spinner in place of the list.
* You provide comments to explain each part of the architecture.
* The code is concise, avoiding unnecessary complexity.
* Do not include unit tests in the code. Return the complete code as a single file that can be copied directly and run in an existing XCode project
Did not compile ❌, required 5 line changes to compile
You will act as an expert Swift developer specializing in The Composable Architecture (TCA) pattern - https://github.com/pointfreeco/swift-composable-architecture
I need you to write Swift code for a SwiftUI feature using TCA that includes:
* A TextField where the user can input a topic.
* A Button that, when tapped, triggers an action to load questions related to the entered topic.
* A List that displays all the questions fetched from a mock API based on the topic.
* A Loading Spinner (ActivityIndicator) that appears while the questions are being fetched.
* If no questions are found, the list should appear empty.
Use The Composable Architecture (TCA) structure, ensuring that the code is:
* Modular, clean, and testable.
* Has a SwiftUI View called 'TopicInputView' that has one text field to enter a topic, and one button to load questions using the text entered by the user.
* The State of the view should be maintained by a TopicInputFeature Reducer, which is a struct that contains the reducer logic, handles how actions modify the state, including setting loading state, handling results, and updating the view accordingly.
* Within the TopicInputFeature struct
* State: Holds the user's topic input, the loading state, the fetched questions, and a flag for handling empty results.
* Action: Triggers for text input changes, button taps, starting the API call, receiving the API response, and finishing loading.
* The API call should be made by a TriviaAPIClient that can be used by the Reducer as a dependency
Ensure that:
* If the API call returns no questions, display an empty list with no error message.
* While the API call is in progress, show a loading spinner in place of the list.
* You provide comments to explain each part of the architecture.
* The code is concise, avoiding unnecessary complexity.
* Do not include unit tests in the code. Return the complete code as a single file that can be copied directly and run in an existing XCode project
Can you modify the code to put the reducer functionality inside a struct called TopicInputFeature
Did not compile ❌, could not change to compile ❌
See MS Copilot Generated Code Here
Scoring Criteria | Cursor Claude 3.5 Sonnet (2nd attempt) |
Chat GPT-4 Swift Copilot GPT |
---|---|---|
Has View, Feature, Client Dependency | 1 | 1 |
View - topic is bound to feature state | 1 | 1 |
Feature - uses “@Reducer” macro | 0 | 0 |
Feature - uses “@ObservableState” macro | 0 | 0 |
Feature uses “var body: some ReducerOf |
1 | 0 |
Feature has “@Dependency” | 1 | 1 |
API Client extends “DependencyKey” | 1 | 0 |
API Client uses struct API pattern | 1 | 1 |
API Client uses “async throws” and returns a Model (not Result) | 1 | 1 |
Score | 77.78% | 55.56% |