“I think I need to go lie down,” posted designer Kevin Cannon in the beginning following the X virus that revealed the creation of function sliders that turns the sound on the screen, the sound is visual for it changing colorsand game works of tic-tac-toe. Soon, others followed with the introduction of a clone Break outcreating a the call clock works that sign, on the snake gamemaking a Pong rewardmeaning a visual state chartand a lot more.
Users can Experiment with a live demo of Make It Real online. However, its operation requires the provision of an API key from OpenAI, which is a security risk. If others access your API key, they can use it to collect huge amounts of money in your name (OpenAI the costs by the amount of data going in and out of your API). Those who are technically inclined can run the code locally, but will still require access to the OpenAI API.
Tldraw, developed by Steve Ruiz in London, is an open source collaborative whiteboard tool. It offers a basic unlimited canvas for drawing, text, and media without requiring a login. Launched in 2021, the project get $2.7 million in seed funding and is supported by GitHub sponsors. When the GPT-4V API was recently launched, Ruiz integrated a prototype called “cause-a-ui“created by Sawyer Hood to bring AI-powered functionality into tldraw.
GPT-4V is a version of OpenAI’s large language model that can interpret visual images and use them as vectors. Like AI expert Simon Willison explain on X, make it a real task by “generating the base64 PNG code of the graphics components, then transferring that to GPT-4 Vision” with the program and instructions to convert the image into a file using Tailwind. In fact, here’s the full program specification that tells GPT-4V how to take the inputs and turn them into working code:
const systemPrompt=”You are an expert web designer who specializes in style css.
A user will give you a small wireframe of an application.
If you have any images, upload them from Unsplash or use solid color corners.
The user will give you notes in blue or red text, arrows, or drawings.
The user may include images of other websites as physical references. Picking styles as best as you can, matching fonts/colors/backgrounds.
They can also give you the html of the previous design they want you to review from.
Make any changes they ask of you.
In the wireframe, the pre-designed html will appear as a white rectangle.
Use creative license to make the material more meaty.
As more people experiment with GPT-4V and combine it with other protocols, we will likely see more novel applications of OpenAI vision technology emerging in the coming weeks. Also on Wednesday, a developer used the GPT-4V API to create a live, real-time video feed by AI-generated voice David Attenborough, which we’ve covered separately.
Right now, it feels like we’ve been given a preview of a possible future state of software development—or visual design, at least—where creating a functional design is as simple as creating a visual mockup and having an AI. The model does the rest. According to developer Michael Dubakov wrote when showing his own creation Make It Real, “OK, @tldraw is officially crazy. It’s really interesting where we’ve ended up in 5 years… I can’t keep up with the pace of innovation anymore.”