Point at the page and say the change

You can now change a page by pointing at it and talking. In Shuriken's browser, turn on Design, click the element you mean and say what should change. The chat's agent gets your words, that element's details and a cropped screenshot of it. If the agent is busy, it goes to the queue instead.

Why we put a microphone on the pointer#

When we shipped Design mode, the clicking part felt right straight away. The typing part didn't. We'd click a button, then sit there composing a sentence about it, which felt like going back to describing the page in words.

Watch someone review a UI with a colleague and they don't type. They point and talk: "this one, bigger", "this gap is too wide", "same as the card above". Fast, a bit messy, and perfectly clear, because the finger already did the hard part.

So in 5.1 we joined two things Shuriken already had: the pointer from the browser, and voice from the composer.

How it works#

1. Open your app in the browser. Pick one of the project's dev servers that Shuriken found, or type an address. The page opens beside your chat.
2. Turn on Design. A strip says "Click the part of the page you want to change." Hold Shift to pick several elements.
3. Press the microphone and talk. "Make this full width on phones and give it a bit more padding." You see the words while you speak.
4. Stop, check, send. Press stop and the words land in the note. Edit anything the speech-to-text got wrong, then Send.

You can also start talking before you've clicked anything. The words wait, and the next element you pick takes them. Point, talk, point at the next thing, keep talking.

What the agent receives#

For every element you picked, the message carries:

• your words, as the message;
• the element's details: where it sits in the page and its markup, so the agent can find it in your code;
• a cropped screenshot of that element, so it sees what you saw.

That combination is the point. "Make this full width" is useless on its own and perfectly clear with a picture of "this" attached. The agent can start on the change instead of asking which button you meant.

If the chat's agent runs in a terminal beside the browser instead, Shuriken pastes the note into that terminal and saves the screenshots as files it can open.

Agent busy? It goes to the queue#

You don't have to wait for a turn to end. While the agent is working, the note's button reads Add to queue. The note joins the chat's queue with its element and screenshot, and runs when the current turn is done. Click the next thing, say the next change, queue it too.

If your connection drops at the wrong moment, the note says "It sends when the connection is back" and keeps it.

Which speech-to-text it uses#

The same as voice everywhere else in Shuriken. By default NVIDIA Parakeet turns what you say into text on your own computer: there's no key to add, nothing to pay per minute, and with the local model no speech service hears the note. If you'd rather use a cloud model, add your own key for a live provider such as ElevenLabs or AssemblyAI in Settings › Dictation, and your notes stream to it under your account.

If you like dictating long prompts too, Your voice stays on your computer by default covers the composer side.

Desktop app only#

Design mode lives in Shuriken's browser, and the browser is part of the desktop app for macOS, Windows and Linux. On the phone you can still talk to your agents through the composer, but you can't point at a page.

A few things stay the way they were. Password and secret fields never leave the page. Every project shares one browser profile, so a site you signed in to once stays signed in. Your agent can drive the same tab you're looking at, so you can watch it click through the fix after you've asked for it. The browser page has the details.

A small habit that helps#

Say the why, not just the what. "Full width on phones, because the tap target is too small" gets you a better change than "full width". The agent reads intent as well as instructions, and talking makes the why easy to include. It's one extra sentence, and you don't have to type it.

Point, talk, send: it comes with Shuriken 5.1. Try Shuriken.