Pairing with GPT-4

Author

Name: Brad Gessler
@bradgessler: @bradgessler

GPT-4 can be helpful for beginner and senior Ruby developers, but it does have limitations. It won’t write all of your software for you, but it will point you in a useful direction, especially if you prefer learning by doing.

Let’s look at how well GPT-4 pairing works by picking an easy, but less well known project with some edge cases: downloading a RubyGem, parsing the docs via YARN, and dumping them into a Sqlite database.

The initial prompt

Create a Ruby script for me that downloads a RubyGem, parses the docs via YARD, and dumps them into a Sqlite database that’s named #{gem-name}-#{gem-version}.db. The database should also include information from the Gem spec.

GPT-4 usually starts its responses with an overview of what it’s about to do, then shows how it solves the problem, followed by recapping and summing up the solution.

GPT-4 response

To create a Ruby script that downloads a RubyGem, parses the documentation using YARD, and stores the data in a SQLite database named with the gem name and version, you will need to follow these steps:

Install the required gems:
- gem install yard
- gem install sqlite3
- gem install bundler
Create a new file named gem_doc_parser.rb and paste the following code:

    require 'yard'
require 'sqlite3'
require 'bundler'

def download_and_parse_gem(gem_name, gem_version)
  gem_path = "#{gem_name}-#{gem_version}"

  # Download the gem
  Bundler::CLI.start(["gem", gem_name, "--version", gem_version])

  # Parse the gem documentation using YARD
  YARD::Registry.clear
  YARD::Parser::SourceParser.parse(gem_path)
end

def create_database(gem_name, gem_version)
  db = SQLite3::Database.new("#{gem_name}-#{gem_version}.db")

  # Create tables for storing gem spec and documentation
  db.execute <<-SQL
    CREATE TABLE IF NOT EXISTS gem_spec (
      id INTEGER PRIMARY KEY,
      name TEXT,
      version TEXT
    );
  SQL

  db.execute <<-SQL
    CREATE TABLE IF NOT EXISTS gem_docs (
      id INTEGER PRIMARY KEY,
      path TEXT,
      type TEXT,
      doc TEXT
    );
  SQL

  db
end

def store_gem_spec(db, gem_name, gem_version)
  db.execute("INSERT INTO gem_spec (name, version) VALUES (?, ?)", [gem_name, gem_version])
end

def store_docs(db)
  YARD::Registry.all.each do |object|
    path = object.path
    type = object.type
    doc = object.docstring.to_raw

    db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
  end
end

def main
  puts "Enter the gem name:"
  gem_name = gets.chomp

  puts "Enter the gem version:"
  gem_version = gets.chomp

  download_and_parse_gem(gem_name, gem_version)

  db = create_database(gem_name, gem_version)
  store_gem_spec(db, gem_name, gem_version)
  store_docs(db)

  puts "Documentation has been stored in #{gem_name}-#{gem_version}.db"
end

main

  

This script will perform the following tasks:

Download and parse the gem documentation using YARD
Create a SQLite database named with the gem name and version
Create tables for storing gem spec and documentation
Store the gem spec (name and version) in the database
Store the documentation in the database

To run the script, simply execute ruby gem_doc_parser.rb and follow the prompts to enter the gem name and version. The documentation will be stored in the specified SQLite database.

Let’s see if it works!

We have GPT-4’s solution. Now its time to copy the code above into gem_doc_parser.rb and run it.

ruby gem_doc_parser.rb

Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
Enter the gem name:
rake
Enter the gem version:
13.0.6
gem_doc_parser.rb:9:in `download_and_parse_gem': uninitialized constant Bundler::CLI (NameError)

  Bundler::CLI.start(["gem", gem_name, "--version", gem_version])
         ^^^^^
  from gem_doc_parser.rb:61:in `main'
  from gem_doc_parser.rb:70:in `<main>'

Uh oh! We already run into our first error. Let’s paste that output into GPT-4 and see what it tells us.

That didn’t work! I got this error…

GPT-4 response

I apologize for the confusion. I misunderstood your initial request. You want to download and parse a specific RubyGem, not create a new gem.

Please follow these steps to update the script:

Replace the download_and_parse_gem method with the following:

    def download_and_parse_gem(gem_name, gem_version)
  gem_path = Bundler::CLI::Common.select_spec(gem_name, gem_version).full_gem_path

  # Parse the gem documentation using YARD
  YARD::Registry.clear
  YARD::Parser::SourceParser.parse(gem_path)
end

  

Install the bundler gem by running:

gem install bundler

Update your Gemfile to include the gem you want to download and parse:

source 'https://rubygems.org'

gem 'rake', '13.0.6'

Install the specified gem with Bundler:

bundle install

After making these updates, you should be able to run the script with ruby gem_doc_parser.rb. The script will parse the documentation for the gem specified in the Gemfile and store it in a SQLite database.

Will GPT-4’s fix work?

Ok, let’s do this one more time just for fun to see if it runs.

    Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
Enter the gem name:
rake
Enter the gem version:
13.0.6
gem_doc_parser.rb:6:in `download_and_parse_gem': uninitialized constant Bundler::CLI (NameError)

  gem_path = Bundler::CLI::Common.select_spec(gem_name, gem_version).full_gem_path
                         ^^^^^^^^
  from gem_doc_parser.rb:58:in `main'
  from gem_doc_parser.rb:67:in `<main>'

  

Nope!

This is where GPT-4 can be problematic for people who might be new to software development that are trying to solve specific problems right away. If you’re using it this way, you’ll find that GPT-4 will give you a plausible answer, but it usually won’t work on the first try.

That’s OK though, let’s get more specific about how we want this to run.

More experience helps GPT-4 give better results

If you’ve built applications before, you can give GPT-4 a more specific prompt and you’ll get back an answer that’s more inline with what you want.

That doesn’t work either. Let’s try this instead: instead of using Bundler to install the gem, just download the .gem file directly from RubyGems, save it to a temporary path, then have Yard document that one gem and dump it into a sqlite file.

Put all of this into one class that I can call from Ruby code. Then let’s call that class from the CLI so that looks something like this:

$ ruby gem-to-sqlite rake 13.0.6

This type of prompt is something that feels more like an experienced developer talking to a developer who might just be starting out. It gives us back some more reasonable code.

    require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'

class GemToSqlite
  def initialize(gem_name, gem_version)
    @gem_name = gem_name
    @gem_version = gem_version
  end

  def download_gem
    gem_url = "https://rubygems.org/downloads/#{@gem_name}-#{@gem_version}.gem"
    uri = URI(gem_url)
    file_path = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}.gem")

    File.write(file_path, Net::HTTP.get(uri))
    file_path
  end

  def extract_gem(file_path)
    spec = nil
    Gem::Package.new(file_path).spec.files.each do |file|
      YARD::Registry.clear
      YARD::Parser::SourceParser.parse_string(file)
      spec = Gem::Package.new(file_path).spec
    end
    spec
  end

  def create_database
    db = SQLite3::Database.new("#{@gem_name}-#{@gem_version}.db")

    db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS gem_spec (
        id INTEGER PRIMARY KEY,
        name TEXT,
        version TEXT
      );
    SQL

    db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS gem_docs (
        id INTEGER PRIMARY KEY,
        path TEXT,
        type TEXT,
        doc TEXT
      );
    SQL

    db
  end

  def store_gem_spec(db, spec)
    db.execute("INSERT INTO gem_spec (name, version) VALUES (?, ?)", [@gem_name, @gem_version])
  end

  def store_docs(db)
    YARD::Registry.all.each do |object|
      path = object.path
      type = object.type
      doc = object.docstring.to_raw

      db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
    end
  end

  def run
    gem_path = download_gem
    spec = extract_gem(gem_path)

    db = create_database
    store_gem_spec(db, spec)
    store_docs(db)

    puts "Documentation has been stored in #{@gem_name}-#{@gem_version}.db"
  end
end

  

When I run it I get another error:

fly-apps/gpt4 → ruby gem_to_sqlite.rb rake 13.0.6
Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
[warn]: Syntax error in `(stdin)`:(1,8): no .<digit> floating literal anymore; put 0 before dot
[warn]: ParserSyntaxError: syntax error in `(stdin)`:(1,8): no .<digit> floating literal anymore; put 0 before dot
[warn]: Stack trace:
  /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:601:in `on_parse_error'
  /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:56:in `parse'
  /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:56:in `parse'
  /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:17:in `parse'
  /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/source_parser.rb:442:in `parse'
  /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/source_parser.rb:124:in `parse_string'

It gives me back this fix:

GPT-4 response

Apologies for the issues you’ve encountered. It appears the error is related to the YARD parser when parsing the gem. I’ve updated the code to address this issue.

Update the extract_gem method in the gem_to_sqlite.rb file with the following code:

    def extract_gem(file_path)
  spec = nil
  gem_dir = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}")

  # Extract the gem to a temporary directory
  Gem::Package.new(file_path).extract_files(gem_dir)
  spec = Gem::Package.new(file_path).spec

  # Parse the gem documentation using YARD
  YARD::Registry.clear
  Dir.glob(File.join(gem_dir, '**', '*.rb')).each do |file|
    YARD::Parser::SourceParser.parse(file)
  end

  spec
end

  

An aside: inline the Gem bundle

I also remembered that I want to share this code for this post, so I asked GPT-4 to move the Gems into an inline bundler.

Could you move the require statements up top into an inline bundler block?

GPT-4 response

Sure! You can use an inline Bundler block to manage gem dependencies. Update the gem_to_sqlite.rb file with the following code:

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'yard', '~> 0.9.28'
  gem 'sqlite3', '~> 1.4'
end

require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'

## ... (rest of the GemToSqlite class definition)

It’s impressive that GPT-4 is able to give me a reasonable response for an “aside”. Let’s see if that distracted it because I ran into another bug running this.

Another bug

I ran into another bug! Check it out:

    fly-apps/gpt4 → ruby gem_to_sqlite-2.rb
[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
  in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':23:

  23: private(*FileUtils.instance_methods(false))

[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
  in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':24:

  24: private(*FileUtilsExt.instance_methods(false))

/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/sqlite3-1.6.1-arm64-darwin/lib/sqlite3/statement.rb:41:in `bind_param': can't prepare Symbol (RuntimeError)
  from /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/sqlite3-1.6.1-arm64-darwin/lib/sqlite3/statement.rb:41:in `block in bind_params'

  

GPT-4 response

I apologize for the oversight. It seems that the error is related to the insertion of data into the SQLite database. The issue comes from the YARD object type being a Symbol, which SQLite doesn’t support. To fix this, we need to convert the object type to a String before inserting it into the database.

Update the store_docs method in the gem_to_sqlite.rb file with the following code:

def store_docs(db)
  YARD::Registry.all.each do |object|
    path = object.path
    type = object.type.to_s
    doc = object.docstring.to_raw

    db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
  end
end

The only change made is converting the object type to a string using to_s:

type = object.type.to_s

With this change, you should be able to run the script without any issues:

./gem-to-sqlite rake 13.0.6

Let’s try it again!

    fly-apps/gpt4 → ruby gem_to_sqlite-2.rb
[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
  in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':23:

  23: private(*FileUtils.instance_methods(false))

[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
  in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':24:

  24: private(*FileUtilsExt.instance_methods(false))

Documentation has been stored in rake-13.0.6.db

Success! Let’s open up the sqlite database and make sure it has what we expect.

sqlite> select * from gem_spec;
1|rake|13.0.6
sqlite> select * from gem_docs;
1|RDoc|module|
2|RDoc::Page|module|
3|RDoc::Page::FONTS|constant|
4|RDoc::Page::STYLE|constant|
5|RDoc::Page::XHTML_PREAMBLE|constant|
6|RDoc::Page::HEADER|constant|
7|RDoc::Page::FILE_PAGE|constant|
8|RDoc::Page::CLASS_PAGE|constant|
9|RDoc::Page::METHOD_LIST|constant|
10|RDoc::Page::FOOTER|constant|
11|RDoc::Page::BODY|constant|
12|RDoc::Page::SRC_PAGE|constant|Source code ##########################
13|RDoc::Page::FR_INDEX_BODY|constant|Index ################################
14|RDoc::Page::FILE_INDEX|constant|
15|RDoc::Page::CLASS_INDEX|constant|
16|RDoc::Page::METHOD_INDEX|constant|
17|RDoc::Page::INDEX|constant|
18|Rake|module|--
Copyright 2003-2010 by Jim Weirich (jim.weirich@gmail.com)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
++
19|Rake::CommandLineOptionError|constant|

Not bad! We can see our gem name and version in the gem_spec table and all of the docs in gem_docs.

Let’s have GPT-4 clean this up for a demo

Since I need to bundle all of this up in a way that I can share it, I’m going to ask GPT-4 to do the work.

It works! I need to get all of this into one file so that I can share it via a Github Gist. Could you consolidate into one file, remove any unnecessary code, and include in code comments up top how to run the script? Make sure its a .rb file and let’s tell people to run it via ruby gem-to-sqlite.rb rake 13.0.6 (for example)

GPT-4 response

Sure! Here’s the consolidated version of the script as a single file named gem-to-sqlite.rb. I’ve also added comments at the beginning of the file with instructions on how to run it.

    # gem-to-sqlite.rb
#
# Usage: ruby gem-to-sqlite.rb <gem_name> <gem_version>
# Example: ruby gem-to-sqlite.rb rake 13.0.6

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'yard', '~> 0.9.28'
  gem 'sqlite3', '~> 1.4'
end

require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'

class GemToSqlite
  def initialize(gem_name, gem_version)
    @gem_name = gem_name
    @gem_version = gem_version
  end

  def download_gem
    gem_url = "https://rubygems.org/downloads/#{@gem_name}-#{@gem_version}.gem"
    uri = URI(gem_url)
    file_path = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}.gem")

    File.write(file_path, Net::HTTP.get(uri))
    file_path
  end

  def extract_gem(file_path)
    spec = nil
    gem_dir = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}")

    # Extract the gem to a temporary directory
    Gem::Package.new(file_path).extract_files(gem_dir)
    spec = Gem::Package.new(file_path).spec

    # Parse the gem documentation using YARD
    YARD::Registry.clear
    Dir.glob(File.join(gem_dir, '**', '*.rb')).each do |file|
      YARD::Parser::SourceParser.parse(file)
    end

    spec
  end

  def create_database
    db = SQLite3::Database.new("#{@gem_name}-#{@gem_version}.db")

    db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS gem_spec (
        id INTEGER PRIMARY KEY,
        name TEXT,
        version TEXT
      );
    SQL

    db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS gem_docs (
        id INTEGER PRIMARY KEY,
        path TEXT,
        type TEXT,
        doc TEXT
      );
    SQL

    db
  end

  def store_gem_spec(db, spec)
    db.execute("INSERT INTO gem_spec (name, version) VALUES (?, ?)", [@gem_name, @gem_version])
  end

  def store_docs(db)
    YARD::Registry.all.each do |object|
      path = object.path
      type = object.type.to_s
      doc = object.docstring.to_raw

      db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
    end
  end

  def run
    gem_path = download_gem
    spec = extract_gem(gem_path)

    db = create_database
    store_gem_spec(db, spec)
    store_docs(db)

    puts "Documentation has been stored in #{@gem_name}-#{@gem_version}.db"
  end
end

if ARGV.length != 2
  puts "Usage: ruby gem-to-sqlite.rb <gem_name> <gem_version>"
  exit 1
end

gem_name = ARGV[0]
gem_version = ARGV[1]

gem_to_sqlite = GemToSqlite.new(gem_name, gem_version)
gem_to_sqlite.run

  

That’s pretty darn good, and the gist is available at Github.

GPT-4 likes to fake it until it makes it (or you lose your patience)

Now I’m going to ask it a question that will probably break it using questionably imprecise language:

How could I use an instance of YARD::Registry instead of the .all singleton?

GPT-4 response

You can create a new instance of YARD::Registry and use it to parse and store the documentation. First, create an instance variable @registry in the initialize method:

    def initialize(gem_name, gem_version)
  @gem_name = gem_name
  @gem_version = gem_version
  @registry = YARD::Registry.new
end

  

Bzzt wrong! YARD::Registry is a module. GPT-4 doesn’t necessarily know what the actual Ruby code is behind what it’s telling you—it seems to be matching patterns on something that seems like it should be a class, but it’s not.

Let’s make a version for squirrels 🐿️

Time to get silly! We made this script for squirrels, so let’s ask GPT to rename a few things to make it more understandable to squirrels.

Let’s rename everything to something that would make a squirrel really happy

    # squirrel_gem_nut_collector.rb
#
# Usage: ruby squirrel_gem_nut_collector.rb <gem_name> <gem_version>
# Example: ruby squirrel_gem_nut_collector.rb rake 13.0.6

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'yard', '~> 0.9.28'
  gem 'sqlite3', '~> 1.4'
end

require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'

class SquirrelGemNutCollector
  def initialize(nutty_gem_name, crunchy_gem_version)
    @nutty_gem_name = nutty_gem_name
    @crunchy_gem_version = crunchy_gem_version
    @nut_registry = YARD::Registry.new
  end

  def nut_fetcher
    nut_url = "https://rubygems.org/downloads/#{@nutty_gem_name}-#{@crunchy_gem_version}.gem"
    uri = URI(nut_url)
    temp_nut_path = File.join(Dir.tmpdir, "#{@nutty_gem_name}-#{@crunchy_gem_version}.gem")

    File.write(temp_nut_path, Net::HTTP.get(uri))
    temp_nut_path
  end

  def crack_nut(file_path)
    nut_spec = nil
    cracked_nut_dir = File.join(Dir.tmpdir, "#{@nutty_gem_name}-#{@crunchy_gem_version}")

    # Crack open the gem nut and release the code
    Gem::Package.new(file_path).extract_files(cracked_nut_dir)
    nut_spec = Gem::Package.new(file_path).spec

    # Parse the code with the power of YARD
    @nut_registry.clear
    Dir.glob(File.join(cracked_nut_dir, '**', '*.rb')).each do |file|
      YARD::Parser::SourceParser.parse(file, @nut_registry)
    end

    nut_spec
  end

  def nut_storage
    nut_db = SQLite3::Database.new("#{@nutty_gem_name}-#{@crunchy_gem_version}.db")

    nut_db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS nut_specs (
        id INTEGER PRIMARY KEY,
        name TEXT,
        version TEXT
      );
    SQL

    nut_db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS nut_docs (
        id INTEGER PRIMARY KEY,
        path TEXT,
        type TEXT,
        doc TEXT
      );
    SQL

    nut_db
  end

  def store_nut_specs(nut_db, spec)
    nut_db.execute("INSERT INTO nut_specs (name, version) VALUES (?, ?)", [@nutty_gem_name, @crunchy_gem_version])
  end

  def fill_nut_storage_with_magic(nut_db)
    @nut_registry.all.each do |object|
      nutty_path = object.path
      crunchy_type = object.type.to_s
      docs_of_wisdom = object.docstring.to_raw

      nut_db.execute("INSERT INTO nut_docs (path, type, doc) VALUES (?, ?, ?)", [nutty_path, crunchy_type, docs_of_wisdom])
    end
  end

  def collect_all_the_nuts
    path_to_nut = nut_fetcher
    nut_spec = crack_nut(path_to_nut)

    nut_db = nut_storage
    store_nut_specs(nut_db, nut_spec)
    fill_nut_storage_with_magic(nut_db)
  end
end

if ARGV.size

  

😆

I’ll spare you these prompts, but I can assure you they were hilarious.

How about for a old timey prospector from the gold rush?
Could you give me a cool hip hop version?
Do one for using words a 4 year old would understand

Overall GPT-4 is a useful tool

To sum it up, here’s what you should keep in mind when you add GPT-4 to your set of tools:

You still need to know what you’re doing

If you’re a senior developer, giving GPT-4 more specific prompts results in better responses. It can reason with you through architectural problems or write you code samples; however, many times it doesn’t know the right answer but confidently tells you something anyway.

If you’re a beginner, GPT-4 will give you responses that you might be able to fix and get running, but it’s not the most concise way of doing it. Often times though the code won’t work and you might not know how to ask it to fix it.

It’s great at jumping contexts

GPT-4 shines when spanning different contexts. If comfortable in one programming language or framework and need to learn a new one, GPT-4 can help explain it to you and relate it back to concepts that you’re accustomed to. In a lot of ways, GPT-4 is like looking into a mirror.

It makes up seemingly plausible answers that could be wrong.

When I asked GPT-4 how to create an instance of a YARD::Registry, instead of using the singleton, it told me YARD::Registry.new . That seems right when thinking about the structure of idiomatic Ruby code, but YARD is not, so GPT-4 has a harder time answering the question.

It gets stuck when the “contexts” get too large, like if you’re asking for code from 5 different files

If you give GPT-4 too broad of a problem to solve, it can get lost and forget things. If you ask it to solve a problem for you like “how do I create server-side anchor tags using Turbo and Hotwire”, it will give you an initial answer that seems right across 5+ files. When you start fixing errors in those files, GPT-4 will loose track of some of the files and start giving you suggestions for fixing completely different code.

Next post ↑: Introducing Live Elements
Previous post ↓: MRSK vs Fly.io